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ABSTRACT 


The Naval Systems Engineering Guide (NESG) was written in 2004 by 
representatives from NAVSEA, NAVFAC, NAVAIR, NAVSUP and 
MARCORSYSCOM. Since then, three other foundational systems engineering 
documents have been written or revised: the International Standards 
Organization (ISO) Standard 15288 in 2008, the International Council on 
Systems Engineering (INCOSE) Systems Engineering Handbook in 2011, and 
The guide to the Systems Engineering Body of Knowledge (SEBoK) in 2012. 
This thesis compares the treatment of one life cycle element, in-service 
engineering, across all four documents, in order to offer a comparative analysis 
of the NESG in light of key industry standards and make recommendations for 
the revision of the NESG. The gap analysis performed in relation to crucial 
industry documents suggests that future revisions of the NSEG should include 
substantial discussion of Operations, Maintenance, Service Life Extension and 
Disposal Processes, as well as identify best practices for each process. 


v 



THIS PAGE INTENTIONALLY LEFT BLANK 


VI 



TABLE OF CONTENTS 


I. INTRODUCTION.1 

A INTRODUCTION TO SYSTEMS ENGINEERING.1 

1. Problem Definition.1 

2. Military Integration.2 

3. Need for Unified Standards.3 

4. Previous Work in this Area.5 

B. HISTORY OF THE NAVAL SYSTEMS ENGINEERING GUIDE.6 

1. MIL-STD 499 and DSMC.6 

2. EIA/IS 632.7 

3. NSEG.7 

C. INDUSTRY STANDARDS FOR SYSTEMS ENGINEERING.8 

1. INCOSE.8 

2. ISO.9 

3. Systems Engineering Body of Knowledge.11 

II. METHODOLOGY.13 

A. AREAS OF COMPARISON.13 

1. In-service Engineering.13 

B. DATA GATHERING PLAN.14 

1. NSEG Research.14 

2. Industry Standards Research.14 

C. DATA ANALYSIS PLAN.15 

1. Standard Comparison.15 

D. CONCLUSIONS AND RECOMMENDATIONS.15 

1. Conclusions.15 

2. Recommendations.15 

III. INDUSTRY STANDARDS DATA GATHERING.17 

A. DEFINITION OF TERMS.17 

1. System Operations.17 

2. System Maintenance.17 

3. System Upgrades.19 

4. System Disposal.20 

B. NSEG.21 

1. Summary.21 

C. ISO 15288.22 

1. System Operations.22 

2. System Maintenance.23 

3. System Upgrades.25 

4. System Disposal.25 

5. Summary.26 

D. INCOSE SE GUIDE.27 

1. System Operations.27 

vii 












































2. System Maintenance.30 

3. System Upgrades.32 

4. System Disposal.33 

5. Summary.36 

E. SEBOK.36 

1. System Operations.36 

2. System Maintenance.39 

3. System Upgrades.40 

4. System Disposal.45 

5. Unique Features.49 

6. Summary.50 

IV. STANDARD COMPARISON.51 

A. NSEG VS. INDUSTRY STANDARDS.51 

1. Introduction.51 

2. Operations Summaries.52 

3. Maintenance Summaries.53 

4. Service Life Extension Summaries.54 

5. Disposal Summaries.55 

B. INCOSESE HANDBOOK VS ISO 15288.56 

1. Operations.56 

2. Maintenance.57 

3. System Life Extension.57 

4. Disposal.57 

C. ISO 15288 VS SEBOK.58 

1. Operations.58 

2. Maintenance.58 

3. System Life Extension.59 

4. Disposal.59 

D INCOSESE HANDBOOK VS. SEBOK.60 

1. Operations.60 

2. Maintenance.60 

3. System Life Extension.60 

4. Disposal.60 

V. CONCLUSIONS & RECOMMENDATIONS.63 

A. INTRODUCTION.63 

1. Introduction.63 

B. RECOMMENDATIONS FOR IMPROVEMENT TO NSEG.63 

1. In-Service Engineering.63 

2. Operations.64 

3. Maintenance.64 

4. Service Life Extension.65 

5. Disposal.65 

C. RECOMMENDATIONS FOR FURTHER STUDY.66 

1. Other Areas of the NSEG.66 

2. Other Standards.66 

viii 
















































3. Continual Revisions.67 

D. CONCLUSIONS.68 

APPENDIX.69 

LIST OF REFERENCES.71 

INITIAL DISTRIBUTION LIST.75 


IX 








THIS PAGE INTENTIONALLY LEFT BLANK 


x 



LIST OF FIGURES 


Figure 1. Diagram from INCOSE SE Handbook, Operations Diagram 

(Haskins, 2011).27 

Figure 2. Diagram from INCOSE SE Handbook, Maintenance Process 

(Haskins, 2011).30 

Figure 3. Diagram from INCOSE SE Handbook, Disposal Process (Haskins, 

2011).33 






THIS PAGE INTENTIONALLY LEFT BLANK 


XII 



LIST OF TABLES 


Table 1. Purpose of SEBoK, table from (Pyster, Olwell, & et-al, System 

Engineering Body of Knowledge, V 0.75, 2012). 12 

Table 2. Comparison Standards Summary.51 

Table 3. Word Count for Standards.52 


xiii 






THIS PAGE INTENTIONALLY LEFT BLANK 


XIV 



LIST OF ACRONYMS AND ABBREVIATIONS 


CUUM: 

Capabilities Update, Upgrades and Modernization 

DoD: 

Department of Defense 

DoN: 

Department of the Navy 

DSMC: 

Defense Systems Management College 

ECSS: 

European Cooperation on Space Standardization 

EIA: 

Electronic Industries Alliance 

INCOSE: 

International Council on Systems Engineering 

ISO: 

International Standards Organization 

MARCORSYSCOM: 

Marine Corps Systems Command 

MOU: 

Memorandum of Understanding 

NASA: 

National Aeronautics and Space Administration 

NAVAIR: 

Naval Air Systems Command 

NAVFAC: 

Naval Facilities Command 

NAVSEA: 

Naval Sea Systems Command 

NAVSUP: 

Naval Supply Command 

NSEG: 

Naval Systems Engineering Guide 

RPG: 

Rocket Propelled Grenade 

SE: 

Systems Engineering 

SEBoK: 

Systems Engineering Book of Knowledge 

SEMG: 

System Engineering Management Guide 

SLE: 

Service Life Extension 

SLEP: 

Service Life Extension Program 


xv 



SPAWAR: 

Space and Naval Warfare Systems Command 

USA: 

United States Army 

USAF: 

United States Air Force 

USMC: 

United States Marine Corps 

USN: 

United States Navy 


XVI 



EXECUTIVE SUMMARY 


This thesis compares the Naval Systems Engineering Guide (NSEG) to 
the International Standards Organization (ISO) Standard 15288, International 
Council on Systems Engineering (INCOSE) Systems Engineering (SE) 
Handbook and the Systems Engineering Body of Knowledge (SEBOK). By 
examining and determining current standards of practice in these key 
documents, focusing on the in-service engineering area of Operations, 
Maintenance, Service Life Extension and Upgrades, and Disposal, this analysis 
identifies areas where updates or revisions to the NSEG can meet or exceed 
industry standards of practice. 

Naval engineers and their contractors need to have up to date standards 
to ensure that they are getting the best equipment possible for the war fighters at 
the best price available; hence, the necessity to determine where the NSEG 
needs to be revised and updated. In addition, due to the long service life of most 
military equipment, there can be significant gains in terms of process, costs and 
efficiency by utilizing SE techniques even years after the systems have been put 
into service. This thesis examined what in-service techniques and methods are 
commonly present in industry standards and should be included in a future 
revision of the NSEG. 

The NSEG does not presently address in-service engineering techniques. 
The NSEG mentions in-service engineering areas in a discussion of the systems 
lifecycle, showing timelines and graphics showing where they should happen. 
This means that any additional focused attention on other dimensions of in- 
service engineering knowledge needs to be drawn from industry standards and 
then modified (as necessary) for Naval purposes. The industry standards that 
were reviewed in this thesis research cover all of the desired areas in the level of 
detail necessary for effective SE: they could easily be adapted to the needs of 
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the Navy. It is recommended that an NSEG revision incorporate content related 
to in-service engineering from the ISO 15288, INCOSE’s Guide to Systems 
Engineering, and the SEBoK. 

The ISO 15288 has bare-bones information on Operations, Maintenance 
and Disposal of a system, but does not cover Service Life Extension and 
Upgrades. The use of this guide as a primary model would be good for the 
NSEG if a minimal amount of direction is desired to be given to contractors and 
Naval engineers. 

The INCOSE SE Handbook has more detail than the ISO 15288 on 
Operations, Maintenance and Disposal of a system, but also does not cover 
Service Life Extension and Upgrades. The INCOSE SE Handbook also includes 
informative summary diagrams that are both useful and effective tools that 
should be included in future revisions of the NSEG. The use of this guide for as 
a primary model would be best if the desired model for the future NSEG was to 
be more informative and offer explicit directions. 

The SEBoK contains extensive detail on Operations, Maintenance, 
Service Life Extension and Upgrades and Disposal. The SEBoK by far has the 
most extensive and lengthiest coverage of the four reviewed standards both for 
the particular areas of interest and for the subject of SE as a whole. The use of 
this standard would be best for the Navy, if the desire is for NSEG to give 
extensive direction and detail on these areas. 

After reviewing all of the standards, this thesis determined that a 
combination of all three industry standards provides the best information for a 
revision to the NSEG. The best attributes for each of the areas of interest are 
summarized below: 

With respect to operations: 

• Adopt an overall Model similar to the one present in INCOSE’s 
guide, with Inputs, Enablers and Controls feeding into Activities that 
give us Outputs. The diagram format that they use is informative 
from a summary purpose and should be included as well. 

• The five elements from this diagram and program would be 
effective if populated from all three manuals. 
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• Adopt the Data Gathering Suggestions from the SEBoK guide, to 
provide the necessary data for studying various operations and 
functions so that improvements based off this data analysis can be 
made for future blocks and/or iterations of the system. 

• Adopt the emphasis on Customer Support and Feedback from ISO 
15288, as having good communication is vital to maintaining a 
healthy relationship between all involved parties. 

With respect to maintenance: 

• Adopt an overall Model similar to the INCOSE Guide, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• The five elements from this diagram and program would be 
effective if populated from all three manuals. 

• Adopt the emphasis on historical records from the ISO 15288 and 
the INCOSE Handbook, as maintaining diligent records will aid in 
making changes and updates to maintenance procedures, tools 
and personnel qualifications. 

• Adopt the emphasis on training that is included in all three manuals. 

• Adopt the emphasis on consistency to establish a pattern for 
various Maintenance activities, as this will allow new methods and 
Operational changes to be made if necessary (i.e., more downtime 
than originally planned or retraining of Operational techniques 
because they are putting undue wear and tear on certain 
components). 

With respect to service life extensions: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• Adopt an emphasis on cost, as in order to modernize or upgrade a 
system, it must be more cost effective than a replacement product 
for the system would be, both short term and long term. 

• Adopt an emphasis on obsolescence. Are the spare parts, tools 
and qualified personnel on hand to keep the products running? If 
not, then upgrades or modernizations may be required to keep the 
system running. 

With respect to system upgrades: 
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• Adopt an overall model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• Adopt an emphasis on cost, as in order to extend a system, it must 
be more cost effective than a replacement product in both the short 
and long term. 

• Adopt an emphasis on obsolescence, to keep the spare parts, tools 
and qualified personnel on hand to keep the products running. 

With respect to disposal: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• The five elements from this diagram and program would be 

effective if populated from all three manuals. 

• Adopt the emphasis on environmental concerns for disposal and 
recycling that are present in all three manuals. 

• Adopt an emphasis on record keeping to preserve data for 

historical reference and as proof that products were properly 

disposed of IAW all applicable laws, standards and regulations. 

• Adopt an emphasis on making Disposal plans part of the process 

from the very beginning, and plan that they will be modified to 

address changes in the product itself and changing laws and 

regulations that govern it over the course of its lifetime. 

By including in-service engineering in the NSEG and by continually 
updating its processes and procedures every few years, the Navy can ensure 
that the NSEG is giving its work force the best tools from a systems engineering 
perspective. This does not guarantee that the best services or systems will make 
it into the hands of war fighters, but it will help to ensure that the best techniques 
are being used. These techniques will ensure that the Navy is getting the proper 
“bang for its buck” and that systems operate more smoothly and efficiently than 
those that do not utilize proper techniques. 
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I. INTRODUCTION 


A INTRODUCTION TO SYSTEMS ENGINEERING 
1. Problem Definition 

Systems Engineering (SE) is a multi-disciplinary study that encompasses 

many fields, ranging from specialized engineering fields (i.e., mechanical, 

electrical, et cetera) to logistics to program management; and it is a field that is 

becoming more crucial as an area of academic and practical study as 

components and systems become more complex and more items are integrated 

into larger networks. For the purposes of this discussion, the definition of SE that 

will be used comes from the INCOSE Systems Engineering Handbook : 

Systems Engineering (SE) is an interdisciplinary approach and means to 
enable the realization of successful systems. It focuses on defining 
customer needs and required functionality early in the development cycle, 
documenting requirements, and then proceeding with design synthesis 
and system validation while considering the complete problem: operations, 
cost and schedule, performance, training and support, test, manufacturing, 
and disposal. SE considers both the business and the technical needs of 
all customers with the goal of providing a quality product that meets the 
user needs. (Haskins, 2011) 

The increased complexity of military hardware, both new systems and 
their integration with legacy systems, requires a correspondingly increased 
expertise in Systems Engineering. Because Systems Engineering covers a wide 
range of topics and areas and these topics all require extensive integration and 
integration oversight, only by utilizing the most current and accepted methods 
and tools can Systems Engineers expect to maintain military hardware in the 
most efficient and cost effective way possible. This thesis seeks to examine how 
SE affects military hardware from a technical standards and methods point of 
view, specifically related to in-service engineering areas in the field. This is a 
major concern for military hardware that often has a long operational life and/or 
has its operational life extended and upgraded extensively during its lifecycle. 
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This thesis examines the Naval Systems Engineering Guide and how it 
compares against three primary industry standards. These standards are the 
International Standards Organizations (ISO) 15288, the International Council on 
Systems Engineering (INCOSE) Systems Engineering Handbook and the 
Systems Engineering Body of Knowledge (SEBoK). The comparison will focus 
primarily on in-service engineering, including Operations, Maintenance, System 
Upgrades and Service Life Extension, and System Disposal. The NSEG does 
not cover in-service engineering techniques, making industry standards the 
primary source material for where to derive content for future revisions to the 
NSEG. 


2 . Military Integration 

Military systems have become increasingly complex and capable. This 
complexity has even reached to the individual U.S. Army soldier or Marine level. 
Where once an infantryman would be equipped with only weapons and body 
armor, he now also carries sophisticated sensor, communications, and 
computing equipment, and may control additional ground and airborne 
unmanned vehicles. Looking at historical data, soldiers have moved from 
approximately 55 lbs. of gear to over 100 lbs. of gear for “marching,” and the 
weight for “assault” operations is an additional 60 lbs of weight, with more gear 
being added as more sophisticated sensing equipment becomes available Little 
of that added weight involves weapons or armor that actually is becoming lighter 
as new technologies and composites are utilized (Task Force Devil, 2003). The 
primary weight increase involves the electronic gear that is being added to 
standard kit. 

Note for a moment that it takes several Human Systems Integration 
Engineers, multiple electronic and computer Engineers, material engineers and 
mechanical engineers to envision and build the items listed above. In order to 
make all these pieces and parts work together, a System Engineer should be 
involved at the various stages of integration, testing and during the lifecycle of 

the equipment. It also conceivable that gear and weapons that are more 
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advanced are being designed for future war fighter use and that these items will 
have to be introduced and integrated as they come online to the soldier “system.” 
These upgrades will be a continual improvement process as war fighters are 
given better tools to do their jobs and to protect themselves from enemy 
combatants. This will require continual process improvement that a Systems 
Engineer will be able to facilitate smoothly. 

If we shift from the single person who is a “grunt” to something more 
complicated and sophisticated, i.e., a warship, submarine or even a jet fighter, 
suddenly these technological advances and integration requirements multiply 
exponentially. Multiple systems engineers are now needed for each of these 
entities, and then an even larger number of systems engineers are required to 
get all of these systems to work together into the “systems of systems” concept. 
The ships, submarines, and aircraft work together to form a battle group and 
each vehicle will be utilizing a variety of technologies at various stages of 
modernization that may have to work together in ways in which they were not 
originally intended. This overall integration project is not a fantasy, it is a daily 
fight that Fleet commanders have to wage as their assets are upgraded and 
changed at different rates. It is imperative Systems Engineers make this 
integration process as simple as possible to aid in Fleet operations and 
engagements. 

All of these issues tie into the next set of questions that emerges as the 
levels of complexity involved in interoperating or systems-of-systems are 
considered: What governs all of these systems and Systems Engineers to ensure 
they are using both best practices and the same practices? Are Naval Systems 
Engineers using the proper standards, techniques and methods when they are 
working? Examining these questions forms the basis for the research conducted 
for this thesis. 

3. Need for Unified Standards 

The DoD does not built or create systems anymore in terms of hardware, 

software or in some cases, concepts, either as components or projects; this has 
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now become the purview of defense contractors, both large and small. Instead, 
the government generates a requirement or need and then goes out to industry 
and asks: Who can meet that need? Answers to this question will come from 
companies of varying size, competency and experience and in some cases, 
there will even be multiple companies building the same item or different 
components of the same item that are all deemed to fit the bill. In order to 
ensure that the multiple companies are building items that meet DoD 
requirements, it is imperative to make certain that they are all using the same 
standard or comparable ones. 

While one would think that there is coherency among those contractors 
and others building and creating to meet DoD needs, with builders and creators 
all working to the same sets of standards and rules for their production, this is 
often not the case. The situation might occur where one company wants to 
utilize ISO standards and one company wants to utilize INCOSE standards on 
two similar projects, and these two projects are supposed to integrate together. 
Will they be able to do that and produce an item to the same performance 
standards? These questions raise an interesting dilemma because SEBoK 
standards were based off of INCOSE standards that were based off of ISO 
standards, making all of these standards related. How close are the “child” 
standards to the “parent”? 

In order for the United States Navy to have a unified Systems Engineering 
Plan, it needs to have an up to date and flexible guide for contractors and for 
Naval Systems Engineers. The NSEG has not been updated since 2004, while 
three major industry standards have been modernized over the last 8 years. In 
January of 2008, ISO published ISO/IEC 15288:2008, which was a replacement 
for ISO 12207 (their SE standard for the previous 10 years) that was intended to 
update all changes made in the SE field, provide backwards compatibility and 
move users towards “harmonized standards” (Roedler & Jones, 2008). In 2011, 
INCOSE published version 3.2.2 of their Systems Engineering Handbook that 
incorporated changes made in the ISO 15288 and is a document targeted 
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specifically at engineers both in and out of the field of SE (Haskins, 2011). In 
2012, SEBoK published their Systems Engineering Body of Knowledge V.75, 
which is intended to be a guide to all items SE related, with the ability to point 
people in the correct direction about particular topics related to different areas of 
SE (Pyster, Olwell, & et-al, System Engineering Body of Knowledge, V 0.75, 
2012). Each of these standards have been through several revisions, with more 
changes likely in the future as SE is a continually evolving and changing field. 

A constant, fluid revision process is necessary for a document to remain 
relevant in the systems engineering world because of the rapid pace of computer 
and technological advances that society is beginning to experience. If the USN 
wants to remain on the cutting edge of technology for both weapons and 
peacetime equipment, it must give its contractors the tools (guides and 
instructions) and flexibility to do business with all options on the table. Using 
updated standards and techniques is one set of tools that are invaluable during 
their creation, modernization, operation and disposal processes. 

4. Previous Work in this Area 

Few documents address topics similar to the line of study that is being 
followed in this document. In 1998 two systems engineers performed a similar 
study of the current military and civilian SE standards and it helped inform 
Systems Engineers what current models should be used at the time (Sheard & 
Lake, 1998). This study not only looked at what was currently in use for SE but 
also which instructions were currently under revision and/or being created for the 
first time. The SMC SE Primer & Handbook is an Air Force document that 
addresses the “basics” of SE for juniors Systems Engineers, Project Officers and 
engineers from other disciplines (Space & Missile Systems Center, 2005). This 
guide is primarily geared toward Space Systems and their associated support 
equipment and processes but the relevant parts to this thesis are the summaries 
of “undeniable facts of SE” and the referral to the INCOSE SE Handbook as a 
primary source of “more-in depth” information (Space & Missile Systems Center, 
2005). 
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B. HISTORY OF THE NAVAL SYSTEMS ENGINEERING GUIDE 

1. MIL-STD 499 and DSMC 

Military Standard 499 was the original DoD SE guide that was written in 
1969 by the USAF and was 32 pages long (USAF Systems Command, 1969). It 
was developed from the USAF AFSCM 375-1 that had been published two years 
earlier and was itself a gathering of knowledge from previous experience and 
work in acquisition and design (Bashaw & Gardella, 1967). The USAF Systems 
Command was responsible for MIL STD-499 and the entire DoD was responsible 
for using it (USAF Systems Command, 1969). That engineering authors wrote a 
standard only 32 pages in length indicates a great transformation has occurred 
between 1969 and the present time in terms of the complexity of systems: MIL- 
STD 499A, written in 1974 also suggests a less complicated field: that version 
only had 24 pages (USAF Systems Command, 1974). These standards only 
contained the broadest definitions of common Systems Engineering and Program 
management terms and sent the readers to other standards, guides and 
technical manuals to get more specific requirements, methods and techniques 
(USAF Systems Command, 1969). 

In 1983, the Defense Systems Management College (DSMC) developed 
their original Systems Engineering Management Guide (SEMG) with a follow on 
update in 1990 (Kockler, Withers, Poodiack, & Gierman, 1990). This guide was 
derived from MIL STD 499A and industry textbooks, and was aimed specifically 
as an introduction to SE from a Program Managers perspective (Kockler, 
Withers, Poodiack, & Gierman, 1990). This guide is over 300 pages long, 
contained significantly more information than MIL-STD 499 series, was focused 
exclusively on defense acquisition programs and was developed exclusively by 
military and DoD defense personnel (Kockler, Withers, Poodiack, & Gierman, 
1990). 

The writing of the DSMC SEMG led to the DoD beginning work on MIL 
STD 499 Revision B (JOINT OSD Working Group, 1993). It was 70 pages long 
and contained significantly more detail than the previous versions of the MIL STD 
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series. This version discussed SE sections and terms in more detail and 
specificity than previous versions, gave specific direction for processes and even 
gave guidance for conducting reviews and creating schedules (JOINT OSD 
Working Group, 1993). This version was never completed or officially published 
and was ultimately replaced by the EIA/IS 632 standard, which was a combined 
effort between government and industry partners (JOINT OSD Working Group, 
1993). 


2. EIA/IS 632 

In June 1994, a working group of industry associations, the International 
Council on Systems Engineering (INCOSE), and the Department of Defense 
developed an interim standard for the engineering of systems; and this effort was 
led by the G-47 Committee on Systems Engineering of the Electronic Industries 
Alliance (EIA) (Department of Navy, 2004). From this working group, a standard 
was written and that standard was the EIA/IS 632 (Department of Navy, 2004) 
(Phillips, 2012). The EIA/IS 632 was intended to provide a standard for use by 
commercial enterprises, as well as government agencies and their development 
contractors and was the replacement for MIL-STD 499B which was still in its draft 
format (Department of Navy, 2004). 

3. NSEG 

The Naval Systems Engineering Guide was published in October 2004 as 
a joint effort by the primary engineering commands of the United States Navy 
(USN) (Department of Navy, 2004). The NSEG was fashioned from the 
Electronic Industries Alliance (EIA) EIA/IS 632 and updating it with inputs from 
NAVSEA, NAVAIR, NAVSUP, MARCORSYSCOM and SPAWAR in order to 
create a document that all could satisfactorily use and implement. The NSEG 
was created in direct response to the directive from the Undersecretary for 
Defense for Acquisition, Technology and Logistics (Wynn, 2004) and was 
implemented by Joint Memorandum of Understanding (MOU) according to the 
following conditions: 
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By reference (a), the Acting Under Secretary of Defense 
(Acquisition, Technology and Logistics) signed into policy the 
requirement that ‘All programs responding to a capabilities or 
requirements document shall apply a robust Systems Engineering 
(SE) approach that balances total system performance and total 
ownership costs within the family-of-systems, system-of-systems 
context. Accordingly, a SE Plan shall be developed for Milestone 
Decision Authority approval in conjunction with each Milestone 
review.’ SE must be embedded in program planning and performed 
across the entire acquisition life cycle. SE provides the integrating 
technical processes to define and balance system performance, 
cost, schedule, and risks. (Massenburg, Langerich, Stone, 
Rodriguez, & Catto, 2004) 

This tasking from the USECDEF AT&L underscores that the entire DoD 
was beginning to take SE seriously. The quick turnaround by the USN (less than 
six months) is an amazingly quick action for a big bureaucracy like the upper 
echelons of the military, which helps enforce the idea that all levels of senior 
government were taking this idea of having to update U.S. Systems Engineering 
practices seriously. The only possibly distressing part of this whole series of 
events is that the NSEG has not been updated since 2004, and eight years is a 
long time when thought of in terms of technology upgrades and changes. This is 
only possibly distressing because that guide may or may not be current and that 
uncertainty is something that will be determined through the course of this 
discussion. 

C. INDUSTRY STANDARDS FOR SYSTEMS ENGINEERING 
1. INCOSE 

The International Council on System Engineering (INCOSE) is a large 
group of academic and professional engineers whose mission is “to share, 
promote and advance the best of systems engineering from across the globe for 
the benefit of humanity and the planet” (Haskins, 2011). With over 8000 
members from over 50 countries and over 20 years since its inception, it is fair to 
say that INCOSE has a good sampling of all sorts of various experiences, 
training and work that enable their knowledge base to be both well-formed and 
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well-respected. The board members from INCOSE include both federal and DoD 
entities, in addition to the many civilian companies which gives their knowledge 
base a wide spread. In addition, INCOSE members actually were part of the 
original EIA/IS 632 working group that was the precursor to the NSEG (INCOSE 
Administration, 2011). 

INCOSE published their first version of a Systems Engineering Guide in 
1994 and the organization has gone through multiple edits, corrections and 
iterations since that time in order to keep its standards up to date. The current 
version is 3.2.2, published in October of 2011, and is nearly 400 pages long 
(Haskins, 2011). The fact that there have been eight revisions to the original 
manual over a 17-year period lends some credit to the fact that SE is a 
continually changing field as people gain a greater understanding and 
appreciation for the discipline. This constant revising process also shows the 
importance of having a dedicated Systems Engineer (not merely a mechanical or 
electrical engineer filling in) working for a company or division whose primary job 
is keeping on top of all the knowledge and changes to the field to ensure 
compliance and use of new tools and methods. 

2. ISO 

The International Standards Organization (ISO) is a universally recognized 
group that publishes standards for a wide range of disciplines, ranging from 
Engineering to Business to Health and Human Services. They advertise 
themselves as publishing a set of “voluntary standards” and have created over 
19,000 of them since being founded in 1947. This group is based out of 
Switzerland and can count members from over 164 countries and from over 3300 
technical groups that aid in their development and manufacturing. While 
compliance with their standards is considered voluntary, most companies take it 
as both a mark of pride and are more likely to be considered as viable business 
partners if they possess the appropriate ISO certification (ISO, 2012). 
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The version of the ISO 15288 systems engineering standard that will be 
examined for this study was published in January of 2008 and is only 81 pages in 
length (Roedler & Jones, 2008). The primary reason for the relatively short 
length of this ISO instruction (and most other ISO instructions) is that ISO 
focuses more on emphasizing the importance of having processes and being 
consistent with those processes, as opposed to actually telling an organization 
how to implement processes (ISO, 2012). According to ISO, the following 
benefits have been gained from utilizing their standards: 

• Cost savings - International Standards help optimize operations 
and therefore improve the bottom line 

• Enhanced customer satisfaction - International Standards help 
improve quality, enhance customer satisfaction and increase 
sales 

• Access to new markets - International Standards help prevent trade 

barriers and open up global markets 

• Increased market share - International Standards help increase 

productivity and competitive advantage 

• Environmental benefits - International Standards help reduce 

negative impacts on the environment 

• Benefits in figures: 

• GBP 2.5 billion - annual contribution standards make to the UK 
economy 

• 80% - percentage of world trade impacted by International 
Standards 

• AUD 100 million - benefits to Australian economy from sampling 

standards in the mining industry 

• 84% reduction in transportation time due to standardization of 

container transport (ISO, 2012) 

These statistics apply to the entire line of ISO standards, not just the SE specific 
ones, but the implication is that all of their standards are winning propositions for 
the organizations that uses them and the organizations with whom they do 
business. 
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3. Systems Engineering Body of Knowledge 

The System Engineering Body of Knowledge (SEBoK) is a relatively 
recent industry standard that was first published in 2010. The SEBoK is a 
collaborative effort led by Stevens Institute of Technology and the Naval 
Postgraduate School that gathers “all systems engineering knowledge” into one 
exhaustive, extensive place in order to ensure that knowledge is available for SE 
professionals. 

The SEBoK is intended to be a guide to the body of knowledge, but 
does not seek to capture all the knowledge directly. It provides 
references to more detailed sources of knowledge, and is 
constructed to facilitate easy update as the field evolves and new 
sources of knowledge emerge. The SEBoK is also intended to be 
global in applicability. Despite the challenge that SE is practiced 
differently from industry to industry and country to country, the 
SEBoK must be useful to systems engineers around the world. The 
authors have been chosen from a diverse set of locales and 
industries to help ensure its broad applicability.” (Pyster, Olwell, & 
et-al, System Engineering Body of Knowledge, V 0.75, 2012) 

The purpose that the SEBoK publishers define for themselves in creating 
this standard is summarized in Table 1. 
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Task Name 

Task Description 

Inform Practice 

Inform systems engineers (glossary) about the 
boundaries, terminology, and structure of their discipline 
and point them to useful information needed to practice 

SE in any application domain (glossary). 

Inform Research 

Inform researchers about the limitations and gaps in 
current SE knowledge that should help guide their 
research agenda. 

Inform Interactors 

Inform performers in interacting disciplines (system 
implementation, project and enterprise (glossary) 
management, other disciplines) of the nature and value 
(glossary) of SE. 

Inform Curriculum Developers 

Inform organizations defining the content that should be 
common in undergraduate and graduate programs in SE. 

Inform Certifiers 

Inform organizations certifying individuals as qualified to 
practice systems engineering. 

Inform SE Staffers 

Inform organizations and managers deciding which 
competencies (glossary) that practicing systems 
engineers should possess in various roles ranging from 
apprentice to expert. 


Table 1. Purpose of SEBoK, table from (Pyster, Olwell, & et-al, System 
Engineering Body of Knowledge, V 0.75, 2012) 


The current version of the SEBoK is VO.75. It was published in March of 
2012, is 678 pages long, and is the second revision in 1.5 years. The editors 
plan to release a final Version 1.0 in September 2012 (Pyster, Olwell, & et-al, 
System Engineering Body of Knowledge, V 0.75, 2012). The SEBoK draws 
heavily from the INCOSE SE handbook and many other industry sources so 
when it is complete, it should be a one stop shopping guide that greatly reduces 
the reference and standard searching that academics, students and industry 
workers perform in their daily SE activities. This guide will ultimately be able to 
give the proper guidance or point someone in the proper direction to gain the 
needed guidance for their research and/or work related to the SE field. 
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II. METHODOLOGY 


A. AREAS OF COMPARISON 

1. In-service Engineering 

As previously stated, the main focus of this discussion will be on Systems 
Engineering for items that are already in-service. One of the first questions that 
comes to mind after reading this purpose would be “Why do we care about SE 
for items that are already in-service?” The answer to this question is twofold: the 
first is that it is never too late to start applying Systems Engineering, even if it is 
to systems already in use, because there are always improvements that can and 
in some cases should be made to systems. Some of the older technologies that 
are still in-service more than likely never had a robust, or in fact, any kind of SE 
plans during their initial construction. Adding SE processes and methods even 
late in the system lifecycle can add significant value to a system from a 
functionality, integration and disposal standpoint. 

The second answer to this question is more complicated and it is that 
many “obsolete” technologies are now being extended beyond their original 
service life and/or being using in ways for which they were not originally designed 
or intended. Ships and airframes that were once only intended to last 30 years 
now have to last 35 or 40, due to funding shortfalls or mission changes. This 
means computer and weapon systems need to be upgraded and integrated into 
new battle group and joint warfare units. Technologies and weapon systems that 
were designed for hunting submarines or aircraft are now being used to hunt 
down drug runners and pirates. 

For example, the USS Nicholas aided in the interdiction of a vessel 
carrying nearly 5,000 pounds of cocaine just two months ago off the coast of 
Columbia (Phillips, 2012) and a month before that, the USS Elrod interdicted 
nearly 1,000 pounds in the Caribbean (U.S. Naval Forces Southern Command 
and U.S. 4th Fleet Public Affairs , 2012). While this capability is obviously in their 
repertoire, chasing down small boats and seizing contraband does not exactly fit 
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in the original mission profile for an anti-submarine warfare frigate with a limited 
air warfare capability. Since the Cold War ended and hunting submarines is not 
as important as it once was, the US Navy management and engineering 
personnel found an additional use for frigates that still have significant hull life left 
(Federation of American Scientists, 2011). Adaption and modification are 
significant parts of SE thinking and methodology that need to be well used and 
available to war fighters with ever changing missions. 

B. DATA GATHERING PLAN 

1. NSEG Research 

The first portion of this chapter involved fact-finding in the NSEG 
particularly in the areas that are associated with in-service engineering 
applications. These areas included Operations, Maintenance, Service Life 
Extension, System Upgrades and finally Disposal. Since this manual is the 
standard that is desired to be updated/changed, it was important to establish a 
baseline on where the NSEG stands. Once it was understood where the NSEG 
stood on in-service SE practices, then it was easier to know what to search for in 
the industry standards. This section also discussed the importance of particular 
areas in the in-service lifecycle to aid in the fact finding for the other standards. 

2. Industry Standards Research 

The next portion of this chapter is an examination of the ISO 15288, 
INCOSE SE Handbook and SEBoK, to determine what was in each manual 
concerning in-service equipment, processes and methods. This information was 
necessary for comparison to the NSEG in the gap analysis process. A gap 
analysis is useful for answering the questions posed in this thesis, as can be 
seen from the following definition of it. A gap analysis is 

“A technique for determining the steps to be taken in moving from a 
current state to a desired future-state. Also called need-gap 
analysis, needs analysis, and needs assessment. Gap analysis 
consists of (1) listing of characteristic factors (such as attributes, 
competencies, performance levels) of the present situation ("what 
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is"), (2) cross listing factors required to achieve the future objectives 

("what should be"), and then (3) highlighting the gaps that exist and 

need to be filled.” (The Business Dictionary, 2012) 

C. DATA ANALYSIS PLAN 

1. Standard Comparison 

Once the in-service SE data was gathered from the four primary sources 
under consideration, each of the three industry standards was compared to the 
NSEG and to each other in order to understand what (if any) are the differences 
that need to be updated and/or considered in the NSEG. Once the comparison 
to the NSEG was completed, the industry standards were compared to one 
another to understand the differences and different takes by each one 
concerning in-service areas. These three industry standards are derivatives of 
one another, so there were no major differences or changes made in each 
successive standard, just some omissions or emphasis changes on one area or 
another. These differences were fully analyzed and explained in conclusions and 
recommendations sections. 

D. CONCLUSIONS AND RECOMMENDATIONS 

1. Conclusions 

After all of the data was gathered and analyzed, conclusions were drawn 
and discussed. These conclusions were relatively straightforward because either 
the standards had the same data requirements and methods as the NSEG and 
each other or they did not. All discrepancies that were discovered were 
addressed and what they actually meant to the revision process was discussed 
here. This section discusses each of the four in-service areas in depth and 
draws conclusions about each one as they relate to all four standards. 

2. Recommendations 

This section consists of two parts. The first set of recommendations is for 

changes that should be made for the next iteration of the NSEG. This was 

determined from the gap analysis that was performed between the NSEG and 
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the three major industry standards. The second set of recommendations are 
what courses of inquiry are suggested for further study. The existing gap 
analysis for the in-service sections of these standards makes it safe to assume 
that it will be necessary to explore the other sections of the NSEG and the other 
three standards to determine the differences in all sections of the manuals. 
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III. INDUSTRY STANDARDS DATA GATHERING 


A. DEFINITION OF TERMS 

1. System Operations 

Systems Operations is the phase in the life cycle that begins after 
Verification and Validation and continues until System Disposal (Department of 
Navy, 2004). This phase covers all level of Operation from the normal operating 
procedures and conditions to the extreme and/or emergency portions of the 
system. Depending on the product, this phase could be considerably longer than 
any other phase of the products life cycle, so it is important to have a robust SE 
plan for this phase (Pyster, Olwell, & et-al, System Deployment and Use, 2012). 

A likely starting point for Operations is a plan for sustained operations and 
this includes coordinating staff and schedules, creating acceptance criteria for 
changes and modifications and coordinating the replacement of legacy systems 
(Roedler & Jones, 2008). Other service considerations can be a range of 
activities, from fuel to transportation to contractor support, and the exact needs 
for these criteria will be determined on a by-project basis (Haskins, 2011). Part 
of this plan will be utilizing trained, qualified personnel and this can be difficult 
because for many new systems, the “right” people may be difficult to identify 
initially and pin down, making this a fluid portion of a project (Haskins, 2011). 
Flexibility will be necessary here, as in all steps of this phase, further 
strengthening the need for SE practices. 

2. System Maintenance 

System Maintenance covers all repair work scheduled and unscheduled 
that needs to be done a system on both a normal and emergency basis (Roedler 
& Jones, 2008). Everything from changing a screw on a set basis to replacing a 
damaged circuit board (an emergency) falls under this “phase” and it should be 
just as long as the Systems Operations Phase (i.e. occurring throughout the 
active lifecycle of the system). This section of the SE guide should include how 
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to account for planned and unplanned maintenance from a logistical, personnel 
and material standpoint (Haskins, 2011). This phase can easily be one of the 
more difficult ones to predict because it is conceivable that for new construction 
there will only be estimates of what will need to be repaired or replaced until a 
large enough sampling of maintenance records can be analyzed (Haskins, 2011). 
Part of this phase will likely include re-examination and rewriting of the technical 
manuals to correct deficiencies or over planned maintenance (Pyster, Olwell, & 
et-al, System Deployment and Use, 2012). 

The first major part of Maintenance is the people involved in the process. 
Having the correct personnel is imperative because just having untrained 
personnel will not get you very far (Haskins, 2011). Personnel need to have the 
necessary maintenance and trouble-shooting skills for the particular equipment 
they are working on and for the tools that they need to use for the maintenance 
they are expected to perform (Haskins, 2011). These personnel will have to work 
within the constraints that they are given (time limits, part limits, etc.) and have 
the competence to provide feedback into the system incase changes or 
recommendations need to be made (Pyster, Olwell, & et-al, System Deployment 
and Use, 2012). 

The constraints the need for maintenance places on a system are often 
part of the initial acceptance/ design parameters (i.e., system will only be down 
10% of the time for scheduled maintenance and 5% of the time for unscheduled 
maintenance). Downtime is not operational time, so having the proper 
constraints and personnel can help keep a system operating as designed. It is 
important that systems are designed for maintainability, because maintenance is 
less likely to be performed if it is too difficult or challenging (Haskins, 2011). By 
gathering data about how maintenance is actually performed, who performs it 
and how long it takes (as opposed to how the system was intended to be 
maintained), changes to maintenance procedures, tools and personnel can be 
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made throughout a system’s lifecycle (Haskins, 2011). In addition to following 
constraints, proper procedures will need to be followed to ensure a system is 
taken care of as the manufacturer intends. 

Maintenance procedures are created for a reason, usually because the 
manufacturer determined that it was the best way to keep a system up and 
operating correctly. In order to keep the system in the best shape it should be 
maintained when and how the manufacturer suggests as best as the operators 
can manage. If there is an obvious flaw or defect found in the maintenance 
procedures, tools or required personnel this should be provided back to the 
manufacturer and program management people so that the proper corrective 
actions can be taken (Haskins, 2011). 

3. System Upgrades 

This phase of Operations really covers two areas: Service Life Extension 
and System/Capability Upgrades (Pyster & Olwell, Product and Service Life 
Management, 2012). Service Life Extension is just as the title sounds; using the 
system for longer than it was planned and built to be in use. The reasons for this 
can be because a replacement system is not yet ready, a system was more 
robust than originally planned, or the system is being re-purposed for another 
task (Pyster & Olwell, Product and Service Life Management, 2012). Regardless 
what reasoning is used, there will need to be an examination of both the 
technical and fiscal aspects of the system to ensure that due diligence was used 
in extending a system’s life (Pyster & Olwell, Product and Service Life 
Management, 2012). 

System Upgrades can include both planned and unplanned changes to 
the original system. A good example of this is a Naval Ship, specifically an 
aircraft carrier that has a hull life of 50+ years and can expect upgrades on most 
weapon, radar and sonar systems especially at the planned nuclear refueling 
period which starts at approximately they 24th year of service life (Federation of 
American Scientists, 2011). Some of these replacements will be planned for 

systems that are in the design or test phase when the ship is completed, and 

19 



other upgrades will come as a result of a new capability or need that is defined 
for the system (Greenert, 2012). The new need may come from a mission 
change for the system or because a new threat in need of being countered has 
been discovered (Greenert, 2012). 

4. System Disposal 

This final phase is important for several reason, the most important 
probably being that the planet has a finite number of resources. Systems need 
to be properly disposed of, both to protect the environment and to maximize the 
potential for recycling (Pyster & Olwell, Product and Service Life Management, 
2012). A system might have hazardous materials (nuclear, chemical or 
biological) that require special considerations to make them safe for disposal and 
this is a consideration that should be built into the entire lifecycle plan, not just 
thought of when a system is being prepped for retirement (Haskins, 2011). The 
recycling portion is a consideration both from a limited resource perspective and 
from a fiscal perspective, and in the currently tight fiscal environment, income 
from recycling is useful. Using a Ship Disposal Project started in 1999, the Navy 
has dropped the price from recycling ships from $1100 per ton to just pennies per 
ton in just over ten years (Team Ships, 2012). 

Having a plan is important in the Disposal process to ensure that a 
disposal has been adequately performed to take care of all possible concerns 
(Roedler & Jones, 2008). Primary concerns would be dangerous or toxic 
material that needs to be disposed of or made safe according to certain 
procedures, or items that contain a government classification or company 
propriety information (Haskins, 2011). There may even be personnel files or data 
in old computer systems that previous users left on the systems and these items 
need to be “wiped” for protection purposes (Pyster & Olwell, Product and Service 
Life Management, 2012). A proper disposal plan will cover all of these concerns 
and more. 

A Disposal Phase needs to follow a comprehensive plan in order to 

ensure that nothing is missed. Following the steps of the actual plan is a good 
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way to ensure that the concerns discussed in the previous sections are 
addressed (Haskins, 2011). Following a systematic process will ensure things 
are accomplished in the order designated to be best for system disposal and to 
be sure nothing is missed. These systematic instructions will likely have been 
planned out many years prior to system disposal so it is conceivable that 
changes may have to be made to the plan along the way for large numbers of the 
same system that have to be disposed of (Pyster & Olwell, Product and Service 
Life Management, 2012). 

Finalizing the disposal is important because having proof of what 
processes and procedures were followed is important for all legal and 
environmental issues that arise from Disposal (Pyster & Olwell, Product and 
Service Life Management, 2012). In addition to these legal concerns, it is good 
to have a history, so the disposal knowledge is available for future projects of a 
similar nature that will at some time need to be disposed of (Haskins, 2011). 
Even if future projects are not similar, if good processes are practiced and can be 
adapted for future disposal, then these historical records have value. 

B. NSEG 

1. Summary 

The NSEG does not specifically address the Systems Operations Phase 
or in fact any in-service topics that are being handled in this report. The NSEG 
covers all phases from the initiation of an idea to the creation, testing and 
deployment of an item but ends with Verification and Validation. The only 
mention of any of the in-service topics is on a few different diagrams that shows 
where they are on the life cycle of a product (Department of Navy, 2004). The 
coverage of the NSEG for in-service engineering is not adequate on any of the 
topics previously listed. 
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C. IS0 15288 

1. System Operations 

ISO 15288 will be the first industry standard that is examined because it is 
the “parent” standard for INCOSE and is the simplest of the three. ISO 15288 
breaks the systems operations phase of the SE down into five phases: Prepare 
for Operations, Perform Operational Activation and Checkout, Use the System 
for Operations, Perform Operational Problem Resolution, and Support the 
Customer (Roedler & Jones, 2008). Each of these major activities has several 
sub-steps that aid in the explanation of SE duties for each particular phase. 

Under the Prepare for Operations, there are three basic sub steps: 

• Prepare a strategy for operation, 

• Obtain other services related to operation of the system 

• Assign trained qualified personnel to be operators (Roedler & 
Jones, 2008). 

Performing Operational Activation and Checkout only has one sub-step 
and that is “activate the system in its intended operational situation to deliver 
instances of service or continuous service according to its intended purpose” 
(Roedler & Jones, 2008). This step tells the operators to operate their systems 
and ensure that all functions perform as they are supposed to and is the “last” 
step prior to the system going into full use and operation. Ideally, any last minute 
corrections or bugs will be discovered at this point to aid in future construction 
and/or prevent any in-service catastrophes. 

Use System for Operations has three sub-steps: 

• Consume materials, as required, to sustain the services; 

• Monitor operation to ensure that the system is operated in 
accordance with the operations plans, in a safe manner and 
compliant with legislated guidelines concerning occupational safety 
and environmental protection; and 

• Monitor the system operation to confirm that service performance is 
within acceptable parameters (Roedler & Jones, 2008). 

Perform Operational Problem Resolution has 3 sub-steps: 
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• Perform failure identification actions when a non-compliance has 
occurred in the delivered services; 

• Determine the appropriate course of action when corrective action 
is required to remedy failings due to changed need; 

• Introduce remedial changes to operating procedures, the 
operational environment, human-machine interfaces and operator 
training as appropriate when human error contributed to failure 
(Roedler & Jones, 2008). 

This section consists of standard troubleshooting steps of finding out what 
went wrong and how to fix it. This can be a human, hardware or procedural 
mistake and it will be the job of the Systems Engineer to figure out exactly what 
went wrong. 

Finally, Support the Customer has one sub-step: Continuously or routinely 
communicate with users to determine the degree to which delivered services 
satisfy their needs (Roedler & Jones, 2008). Communication is the key to 
success in all situations and SE processes are extremely important especially 
from a feedback standpoint. The SE company wants to establish a good 
relationship both for the possibility of repeat or recommended business and from 
a good product standpoint. Most engineers take pride in their work and want to 
help their customers out and if a customer does not communicate that there is a 
problem, then it will be hard for the engineers to be as helpful as they can be. 

2. System Maintenance 

ISO 15288 defines the purpose of System Maintenance as “The purpose 
of the Maintenance Process is to sustain the capability of the system to provide a 
service. This process monitors the system’s capability to deliver services, records 
problems for analysis, takes corrective, adaptive, perfective and preventive 
actions and confirms restored capability.” (Roedler & Jones, 2008). The desired 
outcomes of the system maintenance process are: 

• A maintenance strategy is developed 

• Maintenance constraints are provided as inputs to requirements 

• Replacement system elements are made available 
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• Services meeting stakeholder requirements are sustained 

• The need for corrective design changes is reported and Failure and 
lifetime data is recorded (Roedler & Jones, 2008). 

The guide than further breaks this area down to Maintenance Planning and 

Performance that each have several associated sub-steps. 

Planning the Maintenance has two sub-steps: Prepare a maintenance 
strategy and Define the constraints on system requirements that are unavoidable 
consequences of the maintenance strategy (Roedler & Jones, 2008). Prepare a 
Maintenance strategy is further broken down into four sub-steps: 

• The corrective and preventive maintenance strategy to sustain- 
service in the operational environment in order to achieve customer 
satisfaction; 

• The scheduled preventive maintenance actions that reduce the 
likelihood of system failure without undue loss of services, e.g. 
suspension or restriction of the services 

• The number and type of replacement system elements to be 
stored, their storage locations and conditions, their anticipated 
replacement rate, their storage life and renewal frequency 

• The skill and personnel levels required to effect repairs and 
replacements, accounting for maintenance staff requirements and 
any relevant legislation regarding health and safety, security and 
the environment. These procedures include disassembly strategy, 
fault diagnosis techniques, re-assembly and testing sequences. 
(Roedler & Jones, 2008) 

Performing Maintenance is broken down into eight sub-steps: 

• Obtain the enabling systems, system elements and services to be 
used during maintenance of the system. 

• Implement problem reporting and incident recording to guide 
diagnosis of individual events and histories to support future 
corrective, adaptive, perfective and preventive maintenance. 

• Implement the procedures for correction of random faults and/or 
scheduled replacement of system elements. 

• Initiate corrective action to remedy previously undetected design 
errors. 

• Confirm that logistics actions satisfy the required replenishment 
levels so that stored system elements meet repair rates and 
planned schedules. 
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• Perform preventive maintenance by replacing or servicing system 
elements prior to failure, according to planned schedules and 
maintenance procedures. 

• Perform failure identification actions when a non-compliance has 
occurred in the system. 

• Maintain a history of problem reports, corrective actions and trends 
to inform operations and maintenance personnel, and other 
projects that are creating or utilizing similar system entities. 
(Roedler & Jones, 2008) 

3. System Upgrades 

ISO 15288 does not have a specific section on system upgrades and/or 
service life extension. There are a few feedback “loops” integrated into the 
operations and maintenance procedures for product and process improvement 
but that is all that is currently addressed on this subject. This is an area of study 
and work has looked into extensively at this point by ISO and this could be 
because the phenomena of extensive system life extension and upgrade are 
primarily the concern of the military. 

4. System Disposal 

ISO 15288 defines the purpose of Disposal Process is to end the 
existence of a system entity (Roedler & Jones, 2008). The desired outcomes of 
the Disposal process are: 

• A system disposal strategy is defined. 

• Disposal constraints are provided as inputs to requirements. 

• The system elements or waste products are destroyed, stored, 
reclaimed or recycled. 

• The environment is returned to its original or an agreed state. 

• Records allowing knowledge retention of disposal actions and the 
analysis of long-term hazards are available. 

The disposal process is then further broken down into three different sub-steps: 

Plan the Disposal, Perform the Disposal and Finalize the Disposal. 

Planning the disposal has three sub-steps: 
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• Define a disposal strategy for the system, to include each system 
element and any resulting waste products. 

• Unavoidable constraints on the system design arising from the 
disposal strategy are communicated. 

• Specify containment facilities, storage locations, inspection criteria 
and storage periods if the system is to be stored. (Roedler & Jones, 
2008) 

Performing the disposal has six sub-steps: 

• Acquire the enabling systems or services to be used during 
disposal of a system. 

• Deactivate the system to prepare it for removal from operation. 

• Withdraw operating staff from the system and record relevant 
operating knowledge. 

• Disassemble the system into manageable elements to facilitate its 
removal for reuse, recycling, reconditioning, overhaul, archiving or 
destruction. 

• Remove the system from the operational environment for reuse, 
recycling, reconditioning, overhaul or destruction. 

• Conduct destruction of the system, as necessary, to reduce the 
amount of waste treatment or to make the waste easier to handle 
(Roedler & Jones, 2008). 

Finalize the Disposal has two sub-steps: 

• Confirm that no detrimental health, safety, security and 
environmental factors exist following disposal. 

• Archive information gathered through the lifetime of the system to 
permit audits and reviews in the event of long-term hazards to 
health, safety, security and the environment, and to permit future 
system creators and users to build a knowledge base from past 
experiences (Roedler & Jones, 2008). 

5. Summary 

The ISO 15288 has adequate coverage on the Operations, Maintenance 
and Disposal phases of in-service engineering. Most of the minimum information 
that is needed to perform these functions is presented though there is a lack of 
extensive detail. The major critique of each section is a lack of standard 
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recognition or identification. The ISO 15288 does not address Service Life 
Extension or Upgrades at all so any such information would have to be obtained 
from another standard. 


D. INCOSESE GUIDE 

1. System Operations 


The INCOSE SE Handbook begins its operations section with the 
definition from ISO 15288 as a starting point and then adds some further 
description and a figure to help reinforce the concepts it will be presenting. 
Figure 1 below is a reproduction from the INCOSE SE Handbook. 
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Enablers 

- Organization/Enterprise Policies, 
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• Organization/Enterprise Infrastructure 

- Project Infrastructure 

Operation Enabling Systems _ 


Figure 1. Diagram from INCOSE SE Handbook, Operations Diagram 

(Haskins, 2011) 


As this figure shows, INCOSE breaks down Operations into a series of 
Activities that produce Outputs. Feeding into these Actions there are Inputs, 
Controls and Enablers that shape the Outputs. The expanded lists and some 
explanation when required are listed below. 

The Inputs according to INCOSE are: 


27 




















• Concept Documents - Concept documents generated early in 
the life cycle are used to direct the activities of this process 

• Operator/Maintainer Training 

• Initial Trained Operators and Maintainers - Operation of the 
system includes the humans that will operate, maintain, and 
sustain the system 

• Validated System 

• Validation Report - Including documentation of the validation 

activity results, a record of any recommended corrective 
actions, Design Feedback/Corrective Actions taken, and 
evidence that the system element or system satisfies the 
requirements, or not (Haskins, 2011). 

The enablers and controls are grouped together and they are: 

• Applicable Laws and Regulations 

• Industry Standards - relevant industry specifications and 

standards 

• Agreements - terms and conditions of the agreements 

• Project Procedures and Standards - including project plans 

• Project Directives 

• Organization/Enterprise Policies, Procedures, and Standards - 

including guidelines and reporting mechanisms 

• Organization/Enterprise Infrastructure 

• Project Infrastructure 

• Operation Enabling Systems (Haskins, 2011). 

The desired Outputs for INCOSE are: 

• Operation Strategy - Including staffing and sustainment of 
enabling systems and materials 

• Operation Enabling System Requirements - Requirements for 

any systems needed to enable operation of the system-of- 
interest need to be developed 

• Operation Constraints on Design - Any constraints on the 
design arising from the operation strategy to influence future 
design and specification of similar systems or reused systems- 
elements 

• Operation Report- Including: 
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• System performance reports (e.g., statistics, usage data, 
and operational cost data) 

• System trouble/anomaly reports - with recommendations 
for appropriate action (Haskins, 2011). 

Finally the Activities according to INCOSE are: 

• Prepare for Operation 

• Perform Operational Activation and Check-out 

• Provide operator training and maintain qualified staff 

• Use System for Operations 

• Execute ConOps for the system-of-interest 

• Track system performance and account for operational 
availability 

• Perform operational analysis 

• Perform Operational Problem Resolution 

• Manage operational support logistics 

• Document system status and actions taken 

• Report malfunctions and make recommendations for 

improvement 

• Support the Customer (Haskins, 2011). 

In addition to the format listed above the INCOSE guide also includes “common 
approaches and tips” in each section. In the Operations process, there is one tip 
and it is: 

Depending on the nature of agreements between different 
organizations, the development team may continuously or routinely 
communicate with users to determine the degree to which delivered 
services continue to satisfy their needs. The system may exhibit 
unacceptable performance when system elements implemented in 
hardware have exceeded their useful life or changes in the 
operational environment affect system performance. In the event of 
system failures or anomalies, it may be necessary to conduct 
engineering investigations to identify the source(s) of the failure and 
determine appropriate corrective actions. Systems engineers can 
assist in these activities. (Haskins, 2011) 
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2. System Maintenance 


The INCOSE SE Handbook also begins its Maintenance section with the 
ISO 15288 definition for Maintenance. The standard also has a diagram along 
similar lines to the Operations process with Inputs, Controls and Enablers 


feeding into Actions to produce Outputs that are shown in Figure 2. 


r 
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• Concept Documents 

- Operator/Maintainer Training 
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- Validation Report 
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Outputs 


• Maintenance Strategy 

• Maintenance Enabling System 
Requirements 

- Maintenance Constraints on Design 

- Maintenance Procedure 

- Maintenance Report 


Figure 2. Diagram from INCOSE SE Handbook, Maintenance Process 

(Haskins, 2011) 


The desired Inputs for the Maintenance Process are: 

• Concept Documents - Concept documents generated early in 
the life cycle are used to direct the activities of this process 

• Operator/Maintainer Training 

• Initial Trained Operators and Maintainers - Operation of the 

system includes the humans that will operate, maintain, and 
sustain the system 

• Validated System 

• Validation Report - Including documentation of the validation 

activity results, a record of any recommended corrective 
actions, Design Feedback/Corrective Actions taken, and 
evidence that the system element or system satisfies the 
requirements, or not (Haskins, 2011) 
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The Controls and Enablers for the Maintenance Process are: 

• Applicable Laws and Regulations 

• Industry Standards - relevant industry specifications and 
standards 

• Agreements - terms and conditions of the agreements 

• Project Procedures and Standards - including project plans 

• Project Directives 

• Organization/Enterprise Policies, Procedures, and Standards - 
including guidelines and reporting mechanisms 

• Organization/Enterprise Infrastructure 

• Project Infrastructure 

• Maintenance Enabling Systems (Haskins, 2011) 

• The desired Outputs from the Maintenance Process are: 

• Maintenance Strategy - Accounts for the system’s technical 

availability, replacements for system elements and logistical 

support, maintenance personnel training and staff requirements 

• Maintenance Enabling System Requirements - Requirements 

for any systems needed to enable maintenance of the system- 
of-interest need to be developed 

• Maintenance Constraints on Design - Any constraints on the 
design 

• arising from the maintenance strategy 

• Maintenance Procedure 

• Maintenance Report - Including documentation of the 
maintenance activity results, reporting of failures and 
recommendations for action, and failure and lifetime 
performance data. This report also documents any required 
procedure or system changes that should be accomplished as 
part of on-going configuration management activities (Haskins, 
2011 ) 

There are two main Activities for the Maintenance process and they are: 

• Plan Maintenance 

• Establish a maintenance strategy 

• Define maintenance constraints on the system requirements 
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• Obtain the enabling systems, system elements, and other 
services used for maintenance of the system 

• Monitor replenishment levels of spare parts 

• Manage the skills and availability of trained maintenance 
personnel 

• Perform Maintenance 

• Implement maintenance and problem resolution procedures 
-including scheduled replacement of system elements prior 
to failure (i.e., preventive maintenance) 

• Maintain a history of failures, actions taken, and other trends 
to inform operations and maintenance personnel and other 
projects creating or utilizing similar system elements 

• Monitor customer satisfaction with system and maintenance 
support (Haskins, 2011) 

The common approaches and tips for Maintenance are: 

• Use historic data and performance statistics to maintain high 
levels of reliability and availability and to provide input to 
improve the design of operational and future systems. 

• Planning for maintenance begins early in the system life cycle 
with the development of supportability criteria. These criteria, 
which include reliability and maintainability requirements as well 
as personnel, training, facilities, etc., are included in the defined 
stakeholder requirements or system specification to ensure that 
they are considered in the system design. 

• Maintain configuration management control throughout the 
Utilization and Support Stages in support of the Maintenance 
Process. (Haskins, 2011) 

3. System Upgrades 

The INCOSE SE Handbook also does not have a section dedicated to 
System service life extension and upgrading, but there are a few parts of the 
standard that address some aspects of the process. When describing the 
“support stage” of the system lifecycle INCOSE states: 
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Modifications may be proposed to resolve supportability problems, 
to reduce operational costs, or to extend the life of a system. These 
changes require SE assessment to avoid loss of system 
capabilities while under operation. The corresponding technical 
process is the Maintenance Process. (Haskins, 2011) 

INCOSE SE Handbook also makes provides a framework to determine if a 
products lifecycle should be extended based on costs and obsolescence of the 
product. 


4. System Disposal 


This standard also begins its Disposal section with the disposal definition 
from the ISO standard. The familiar INCOSE style diagram is used with Inputs, 
Controls and Enablers feeding into Activities to produce Outputs and that 
diagram is reproduced in Figure 3. 
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Figure 3. Diagram from INCOSE SE Handbook, Disposal Process (Haskins, 

2011 ) 


The Inputs to the Disposal Process are: 

• Concept Documents - Concept documents generated early in 
the life cycle are used to direct the activities of this process. 
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• Validated System - The Disposal Process works on a depleted 
system of system elements (e.g., batteries) meaning that if 
production and operational environments must be restored to 
former conditions, details of the initial state are relevant. 
(Haskins, 2011) 

The Controls and Enablers for the Disposal process are: 

• Applicable Laws and Regulations 

• Industry Standards - relevant industry specifications and 
standards 

• Agreements - terms and conditions of the agreements 

• Project Procedures and Standards - including project plans 

• Project Directives 

• Organization/Enterprise Policies, Procedures, and Standards - 

including guidelines and reporting mechanisms 

• Organization/Enterprise Infrastructure 

• Project Infrastructure 

• Disposal Enabling Systems (Haskins, 2011) 

The desired Outputs for the Disposal process are: 

• Disposal Strategy 

• Disposal Enabling System Requirements - Requirements for 

any systems needed to enable disposal of the system-of-interest 
need to be developed 

• Disposal Constraints on Design - Any constraints on the design 
arising from the disposal strategy 

• Disposal Procedure 

• Disposed System 

• Disposal Report - Including documentation of the disposal 

activity results, may include an inventory of system elements for 
reuse/storage and any documentation or reporting required by 
regulation or organization standards (Haskins, 2011) 

The Actions that are part of the Disposal Process are: 

• Plan Disposal 

• Review the Concept of Disposal, including any hazardous 
materials and other environmental impacts to be 
encountered during disposal 
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• Define the Disposal Strategy 

• Impose associated constraints on the system requirements 

• Perform Disposal 

• Deactivate the elements to be terminated 

• Disassemble the elements for ease of handling 

• Remove the elements and any associated waste products 
from the operational site - includes removing materials from 
storage sites and consigning the elements and waste 
products for destruction or permanent storage 

• Finalize the Disposal 

• Maintain documentation of all Disposal activities and residual 
hazards (Haskins, 2011) 

The common approaches and tips for the Disposal section are: 

• The project team conducts analyses to develop solutions for 
ultimate disposition of the system, constituent elements, and 
waste products based on evaluation of alternative disposal 
methods available. Methods addressed should include storing, 
dismantling, reusing, recycling, reprocessing, and destroying 
end products, enabling systems, system elements, and 
materials. 

• Disposal analyses include consideration of costs, disposal sites, 

environmental impacts, health and safety issues, responsible 
agencies, handling and shipping, supporting items, and 
applicable federal, state, local, and host-nation regulations. 

• Disposal analyses support selection of system elements and 
materials that will be used in the system design, and should be 
readdressed to consider design and project impacts from 
changing laws and regulations throughout the project life cycle. 

• Disposal Strategy and design considerations are updated 
throughout the system life cycle in response to changes in 
applicable laws, regulations, and policy. 

• Consider donating an obsolete system. - Many items, both 
systems and information, of cultural and historical value have 
been lost to posterity because museums and conservatories 
were not considered as an option during the disposal stage. 

• Concepts such as Zero Footprint and Zero Emissions drive 
current trends toward corporate social responsibility that 
influence decision making regarding cleaner production and 
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operational environments and eventual disposal of depleted 
materials and systems. 

• The ISO 14000 series includes standards for Environmental 
Management Systems and Life-Cycle Assessment. 

• Instead of designing cradle-to-grave products, dumped in 
landfills at the end of their 'life,' a new concept is transforming 
industry by creating products for cradle-to-cradle cycles, whose 
materials are perpetually circulated in closed loops. Maintaining 
materials in closed loops maximizes material value without 
damaging ecosystems (Haskins, 2011). 

5. Summary 

The INCOSE SE Handbook has good-to-adequate coverage of all areas 
of Operations, Maintenance and Disposal areas of in-service engineering. There 
is great emphasis placed on standards for every section and at least some 
mention of the other areas deemed important. The INCOSE SE Handbook is 
unique in that it uses diagrams to summarize its main points and this is viewed 
as a positive experience because pictures help to get points across in ways that 
paragraphs sometimes cannot. There is no coverage of Service Life Extension 
or Upgrades so each that information will have to be obtained from another 
standard. The INCOSE SE Handbook demonstrates what needs to be the case 
as far as systems engineering processes go. However, it does not say how to do 
it, which is something that the NSEG will need to do. 

E. SEBOK 

1. System Operations 

The SEBoK follows some of the similar methodologies for their processes 
that are in the INCOSE SE Handbook for the Systems Operations section. It 
begins with the definition of Operations from the ISO 15288 and then adds some 
additional discussion on the subject. The SEBoK then moves into a process 
approach section and states that data collection and analysis are the main 
functions for SE during this phase (Pyster, Olwell, & et-al, System Deployment 
and Use, 2012). The recommended data to be gathered is: 
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• Updating training and development of new training as required 
for operational and support personnel. Training is generally 
developed early with system design and production and 
executed during integration and operations. Determination of 
training updates or changes will be based on evaluation of the 
operational and support personnel. 

• Evaluation of operational effectiveness. Early in the planning 
phases of a new system or capability, measures of operational 
effectiveness are established based on mission and business 
goals. These measures are important during system operation. 
These attributes are unique for each system and represent 
characteristics describing the usefulness of the system as 
defined and agreed to by system stakeholders. Systems 
engineers monitor and analyze these measurements and 
recommend actions. 

• Failure reporting and corrective actions (FRACA) activities will 
involve the collection and analysis of data during operations. 
FRACA data will provide trends involving failures that may 
require design or component changes. Some failures may also 
result in safety issues requiring operational modifications until 
the offending elements under analysis can be corrected. If 
components or systems must be returned to maintenance 
facilities for corrective repairs, there will be operational and 
business impacts due to increased unavailability and unplanned 
transportation cost (Pyster, Olwell, & et-al, System Deployment 
and Use, 2012). 

The SEBoK then lists applicable Methods and Tools: 

• Operations manuals will provide operators the steps and 
activities required to run the system. 

• Training and Certification. Adequate training must be provided 
for the operators who are required to operate the system. The 
objectives of training are to: 

• Provide initial training for all operators in order to equip them 
with the skill and knowledge to operate the system. Ideally, 
this process will begin prior to system transition and will 
facilitate delivery of the system. It is important to define the 
certification standards and required training materials up 
front (for more information on material supply, please see 
Logistics). 

• Provide continuation training to ensure currency of 
knowledge. 
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• Monitor the qualification/certification of the operators to 
ensure that all personnel operating the system meet the 
minimum skill requirements and that their currency remains 
valid. 

• Monitor and evaluate the job performance to determine the 
adequacy of the training program (Pyster, Olwell, & et-al, 
System Deployment and Use, 2012). 

According to the SEBoK the result of the implementation of a successful 
Operations process will: 

• an operation strategy is defined and refined along the way; 

• services that meet stakeholder requirements are delivered; 

• approved, corrective action requests are satisfactorily 
completed; and 

• stakeholder satisfaction is maintained (Pyster, Olwell, & et-al, 
System Deployment and Use, 2012). 

The SEBoK then lists the Outputs of the Operations Process: 

• operational strategy, including staffing and sustainment of 
enabling systems and materials (this may incorporate the 
strategy first defined during the transition process); 

• system performance reports (statistics, usage data, and 
operational cost data); 

• system trouble/anomaly reports with recommendations for 
appropriate action; and 

• operational availability constraints to influence future design and 
specification of similar systems or reused system elements 
(Pyster, Olwell, & et-al, System Deployment and Use, 2012). 

Finally, the Operations section ends with the Activities of the process: 

• provide operator training to sustain a pool of operators; 

• track system performance and account for operational 
availability; 

• perform operational analysis; 

• manage operational support logistics; 

• document system status and actions taken; and 

• report malfunctions and recommendations for improvement 
(Pyster, Olwell, & et-al, System Deployment and Use, 2012) 
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2. System Maintenance 

The important Considerations for the System Maintenance process 
according to the SEBoK are: 

• Maximizing system availability to meet the operational 
requirements. This has to take into account the designed-in 
reliability and maintainability of the system and resources 
available. 

• Preserving system operating potential through proper planning 
of system scheduled maintenance. This requires a reliability- 
centered maintenance strategy that incorporates preventive 
maintenance in order to preempt failures, thereby extending the 
mean time between corrective maintenance, as well as 
enhancing the availability of the system. 

• Outsourcing non-critical maintenance activities so as to optimize 
scarce technical manpower resources. 

• Harness IT technology for maintenance management. This 
involves rigorous and systematic capturing and tracking of 
operating and maintenance activities to facilitate analysis and 
planning (Pyster, Olwell, & et-al, System Deployment and Use, 
2012 ). 

With the successful implementation of the maintenance process the 
results should be: 

• a maintenance strategy is developed; 

• maintenance constraints are provided as inputs to requirements; 

• replacement system elements are made available; 

• services meeting stakeholder requirements are sustained; 

• the need for corrective design changes is reported; and 

• failure and lifetime data is recorded (Pyster & Olwell, Product 
and Service Life Management, 2012). 

The following actions and tasks should be implemented: 

• system preparation for operations, including system 
performance verification before operation; 

• scheduled servicing, such as daily inspection/checks, servicing, 
and cleaning; 

• unscheduled servicing (carrying out fault detection and isolation 
to the faulty replaceable unit and replacement of the failed unit); 
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• re-configuration of the system for different roles or functions; 

• scheduled servicing (higher level scheduled servicing but below 
depot level); 

• unscheduled servicing (carrying out more complicated fault 
isolation to the faulty replaceable unit and replacement of the 
failed unit); 

• minor modifications; 

• minor damage repairs; 

• major scheduled servicing (e.g., overhaul and corrosion 

treatment); 

• major repairs (beyond normal removal and replacement tasks); 
and 

• major modifications (Pyster, Olwell, & et-al, System Deployment 

and Use, 2012). 

Two final considerations listed by the SEBoK under the maintenance 
section are: 

• Adequate training must be provided for the technical personnel 
maintaining the system. While initial training may have been 
provided during the transition process, additional personnel may 
need to be trained to cope with the increased number of 
systems being fielded, as well as to cater to staff turnover. It is 
important to define the certification standards and contract for 
the training materials as part of the supply agreement. 

• The organization responsible for maintaining the system should 
have clear thresholds established to determine whether a 
change requested by end users, changes to correct latent 
defects, or changes required to fulfill the evolving mission are 
within the scope of a maintenance change or require a more 
formal project to step through the entire systems engineering 
life-cycle. Evaluation criteria to make such a decision could 
include cost, schedule, risk, or criticality characteristics (Pyster, 
Olwell, & et-al, System Deployment and Use, 2012). 

3. System Upgrades 

The SEBoK divides up this section into two areas, Service Life Extension 
(SLE) and Capability Updates, Upgrades and Modernization. The SEBoK 
defines SLE as: 
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Product and service life extension involves continued use of a 
product and/or service beyond its original design life. Product and 
service life extension involves assessing the risks and the life cycle 
cost of continuing the use of the product or service versus the cost 
of a replacement system. Service life extension emphasizes 
reliability upgrades and component replacement or rebuilding of the 
system to delay the system’s entry into wear-out status due to 
prohibitively expensive sustainment, reliability, safety, and/or 
performance requirements that can no longer be met. The goal is 
typically to return the system to as new of a condition as possible 
while remaining consistent with the economic constraints of the 
program. SLE is regarded as an environmentally friendly way to 
relieve rampant waste by prolonging the use-life of retiring products 
and preventing them from being discarded too early when they still 
have unused value. However, challenged by fast-changing 
technology and physical deterioration, a major concern in planning 
a product SLE is considering to what degree a product or service is 
fit to have its life extended. (Pyster & Olwell, Product and Service 
Life Management, 2012) 

The key factors and questions to consider for SLE are: 

• Current life cycle costs of the system; 

• Design life and expected remaining useful life of the system; 

• Software maintenance; 

• Configuration Management; 

• Warranty policy; 

• Availability of parts, subsystems, and manufacturing sources; 
and 

• Availability of system documentation to support life extension 
(Pyster & Olwell, Product and Service Life Management, 2012) 

The largest emphasis in this entire section of the SEBoK is cost, with the 

bottom line being if the cost of SLE is less than the cost of replacement than it is 

a course that should be pursued by the operating/managing agency. Some other 

considerations that are mentioned are the disruption of critical services, the effect 

on systems with-in a system (the NASA space program is the example given in 

the SEBoK) and safety in the case of highly dangerous systems (transportation 

aircraft and nuclear reactors) (Pyster & Olwell, Product and Service Life 

Management, 2012). One of the final warnings given is that for some systems 
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there are regulatory requirements associated with SLE (Pyster & Olwell, Product 
and Service Life Management, 2012). 


The Capability Updates, Upgrades and Modernization section starts out 
once again with a similar message to the SLE section, which is that costs needs 
to be one of the major deciding factors. A list of all the major considerations are: 

• type of system (space, air, ground, maritime , and safety 
critical); 

• missions and scenarios of expected operational usage; 

• policy and legal requirements that are imposed by certain 
agencies or business markets; 

• product or service life cycle costs; 

• electromagnetic spectrum usage expected, including change in 
RF emissions; 

• system Original Equipment Manufacturer (OEM) and key 
suppliers, and availability of parts and subsystems; 

• understanding and documenting the functions, interfaces, and 
performance requirements, including environmental testing and 
validation; 

• system integration challenges posed by the prevalence of 
system-of-systems solutions and corresponding interoperability 
issues between legacy, modified, and new systems; and 

• the amount of regression testing to be performed on the existing 
software (Pyster & Olwell, Product and Service Life 
Management, 2012). 

The Key processes and procedures that need to be considered are: 

• legislative policy adherence review and certification; 

• safety critical review; 

• engineering change management and configuration control; 

• analysis of Alternatives; 

• warranty and product return process implementation; and 

• availability of manufacturing and supplier sources and products 
(Pyster & Olwell, Product and Service Life Management, 2012). 

Specifically for Product Systems SEBoK emphasizes: 
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• product modernization involves understanding and managing a 
list of product deficiencies, prioritized change requests, and 
customer issues associated with product usage. 

• product modernization uses the Engineering Change 
Management principle of change control boards to review and 
implement product changes and improvements. 

• product modernization and upgrades require the use of system 
documentation. A key part of the product change process is to 
change the supporting system documentation functions, 
interfaces, modes, performance requirements, and limitations. 

• if system documentation is not available, Reverse Engineering 
is required to capture the proper “as is configuration” of the 
system and to gain understanding of system behavior prior to 
making any changes. 

• during system verification and validation (after product change), 
it is important to perform regression testing on the portions of 
the system that were not modified to confirm that upgrades did 
not impact the existing functions and behaviors of the system. 
The degree and amount of regression testing depends on the 
type of change made to the system and whether the upgrade 
includes any changes to those functions or interfaces involved 
with system safety. 

• It is important to consider changes to the system support 
environment Change may require modification or additions to 
the system test equipment and other support elements such as 
packaging and transportation. 

• Some commercial products involve components and 
subsystems where modernization activities cannot be 
performed. An example of these types of commercial systems 
are consumer electronics, such as radios and computer 
components. The purchase price of these commercial systems 
is low enough that upgrades are not economical and are 
considered cost prohibitive (Pyster & Olwell, Product and 
Service Life Management, 2012). 

For Service Systems the SEBoK recommends: 

• Service system modernization may require regulatory changes 
to allow the use of new technologies and new materials. Service 
system modernization requires backward compatibility to 
previous provided service capability during the period of 
change. 
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• Service system modernization, which spans large geographical 
areas, requires a phased-based change and implementation 
strategy. Transportation systems such as highways (i.e., 
Interstate Highways) provide service to many different types of 
consumers and span such large geographical areas. 

• Modernization often requires reverse engineering prior to 
making changes to understand how traffic monitoring devices 
such as metering, TV cameras, and toll tags interface with the 
rest of the system (Pyster & Olwell, Product and Service Life 
Management, 2012). 

For Enterprises the SEBoK recommends: 

• Enterprise system modernization must consider the location of 
the modification and the conditions under which the work will be 
performed. The largest challenge is implementing the changes 
while the system remains operational. In these cases, disruption 
of ongoing operations is a serious risk. For some systems, the 
transition between the old and new configuration is particularly 
important and must be carefully planned. 

• Enterprise system modernization requires coordination of 
changes across international boundaries. Enterprise 
modifications normally occur at a lower level of the system 
hierarchy. Change in requirements at the system level would 
normally constitute a new system or a new model of a system. 

• The Chapter Guidebook (2010) discusses the change to the 
architecture of the system. In cases where a component is 
added or changed, this change will constitute a change to the 
architecture. 

• As an example, the global positioning system (GPS) is an 
enterprise system implemented by the US military but used by 
both commercial and government consumers worldwide. 
Modernization may involve changes to only a certain segment of 
the enterprise, such as the ground user segment to reduce size, 
weight, and power. Modernization may only occur in certain 
geographical areas of operation. 

• The air transportation system consists of multiple countries and 
governing bodies dispersed over the entire world. Changes can 
occur locally or can require coordination and integration 
worldwide (Pyster & Olwell, Product and Service Life 
Management, 2012). 
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The next area of this section addresses concurrent modification 
management methods and suggests two different methods if this approach is to 
be utilized: 

• The first method is called the “block” method. This means that a 
group of systems are being modified simultaneously and will be 
deployed together as a group at a specific time. This method is 
meant to ensure that at the end state, all the modifications have 
been coordinated and integrated so there are no conflicts and 
no non-compliance issues with the system-level requirements. 

• The second method is called continuous integration and is 
meant to occur concurrently with the block method. Information 
management systems provide an example of a commercial 
system where multiple changes can occur concurrently. The 
information management system hardware and network 
modernization will cause the system software to undergo 
changes. “Software release management is used to coordinate 
the proper timing for the distribution of system software changes 
to end-users””. (Pyster & Olwell, Product and Service Life 
Management, 2012) 

The final area of this section addresses Commercial Off the Shelf (COTS) 
items and the benefits and risks associated with them. The advantages are that 
COTS items provide increased functionality, continually shrinking size and lower 
typical costs than specialized products. The primary disadvantages for COTS 
items are obsolescence and changes to system interface. SEBoK concludes 
with the thought that extensive consideration needs to be given to form factor 
and electrical and physical interface (Pyster & Olwell, Product and Service Life 
Management, 2012). 

4. System Disposal 

According to the SEBoK, the reasons for systems disposal are that at 
some point a system will become uneconomical to maintain, obsolete or 
unrepairable (Pyster & Olwell, Product and Service Life Management, 2012). 
Some of the concerns for System Disposal listed by the SEBoK are: 

• In addition to technological and economical factors, the system 
being developed must be compatible, acceptable, and ultimately 
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address the design of a system for the environment in terms of 
ecological, political, and social considerations. 

• In particular, the ecological considerations associated with 
system disposal or retirement is of prime importance. The most 
concerning problems of dealing with waste are identified below. 

• Air Pollution and Control 

• Water Pollution and Control 

• Noise Pollution and Control 

• Radiation 

• Solid Waste 

• In the United States, the Environmental Protection Agency 
(EPA) and the Occupational Safety and Health Administration 
(OSHA) govern disposal and retirement of commercial systems; 
equivalent organizations perform this function in other countries 
(Pyster & Olwell, Product and Service Life Management, 2012). 

The listed applications for Product Systems are: 

• Product system retirement may include system disposal 
activities or preservation activities (e.g., mothballing) if there is a 
chance the system may be called upon for use at a later time. 
OSD AT&L provides guidance for the preservation of military 
systems, such as naval ships and aircraft. 

• ““Systems Engineering and Analysis has several chapters that 
discuss the topics of design for goals such as “green 
engineering,” reliability, maintainability, logistics, supportability, 
producibility, disposability, and sustainability. Chapter 16 
provides a succinct discussion of “green engineering” 
considerations and “ecology-based manufacturing.” Chapter 17 
discusses life cycle costing and the inclusion of system disposal 
and retirement costs.”” 

• Some disposal of a system's components occurs during the 
system’s operational life. This happens when the components 
fail and are replaced. As a result, the tasks and resources 
needed to remove them from the system need to be planned 
well before the actual demand for disposal occurs. Planning 
must consider transportation of failed items, handling 
equipment, special training requirements for personnel, 
facilities, technical procedures, technical documentation 
updates, hazardous material (HAZMAT) remediation, all 
associated costs, and reclamation or salvage value for precious 
metals and recyclable components. ““Phase-out and disposal 
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planning addresses when disposal should take place, the 
economic feasibility of the disposal methods used, and what the 
effects on the inventory and support infrastructure, safety, 
environmental requirements, and impact to the environment will 

be..Disposal is the least efficient and least desirable 

alternative for the processing of waste material.”” 

• The EPA collects information regarding the generation, 
management, and final disposition of hazardous wastes 
regulated under the Resource Conservation and Recovery Act 
of 1976 (RCRA). 

• EPA waste management regulations are codified at 40 C.F.R. 
parts 239-282. Regulations regarding management of 
hazardous wastes begin at 40 C.F.R. part 260. Most states 
have enacted laws and promulgated regulations that are at least 
as stringent as federal regulations. Due to the extensive tracking 
of the life of hazardous waste, the overall process has become 
known as the cradle-to-grave system. Stringent bookkeeping 
and reporting requirements have been levied on generators, 
transporters, and operators of treatment, storage, and disposal 
facilities that handle hazardous waste. 

• See the EPA website for a comprehensive list of wastes, 
including resource conservation, hazardous wastes, and non- 
hazardous wastes. 

• Unfortunately, disposability has a lower priority compared to 
other activities associated with product development. This is 
due to the fact that, typically, the disposal process is viewed as 
an external activity to the entity that is in custody of the system 
at the time. Some of the reasons behind this view include: 

• There is no direct revenue associated with the disposal 
process and the majority of the cost associated with the 
disposal process is initially hidden. 

• Typically, someone outside of systems engineering performs 
the disposal activities, thus the "not my problem" attitude is 
common. For example, neither a car manufacturer nor the 
car's first buyer may be concerned about a car's disposal 
since the car will usually be sold before disposal. 

• The European Union’s Registration, Evaluation, Authorization, 
and Restriction of Chemicals (REACH) regulation requires 
manufacturers and importers of chemicals and products to 
register and disclose substances in products when specific 
thresholds and criteria are met (European Parliament 2007). 
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The European Chemicals Agency (ECHA) manages REACH 
processes. 

• Numerous substances will be added to the list of substances 
already restricted under European legislation; for example, a 
new regulation emerged when the Restriction on Hazardous 
Substances (RoHS) in electrical and electronic equipment was 
adopted in 2003. 

• Requirements for substance use and availability are changing 
across the globe. Identifying the use of materials in the supply 
chain that may face restriction is an important system life 
management consideration. System disposal and retirement 
requires upfront planning and the development of a disposal 
plan to manage the activities. An important consideration during 
system retirement is the proper planning required to update the 
facilities needed to support the system during retirement, as 
explained in the California Department of Transportation 
Systems Engineering Guidebook. 

• Disposal needs to take into account environmental and personal 
risks associated with the decommissioning of a system and all 
hazardous materials need to be accounted for. The 
decommissioning of a nuclear power plant is a prime example of 
hazardous material control and exemplifies the need for 
properly handling and transporting residual materials resulting 
from the retirement of certain systems. 

• The Defense Logistics Agency (DLA) is the lead military agency 
responsible for providing guidance for worldwide reuse, 
recycling, and disposal of military products. A critical 
responsibility of the military services and defense agencies is 
demilitarization prior to disposal (Pyster & Olwell, Product and 
Service Life Management, 2012). 

The application for Service Systems are: 

• An important consideration during service system retirement or 
disposal is the proper continuation of services for the 
consumers of the system. As service systems are retired, it is 
often integral to continue to provide the same quality and 
capacity of services offered by the system. As an existing 
service system is decommissioned, a plan should be adopted to 
bring new systems online that operate in parallel of the existing 
system so that service interruption is kept to a minimum. This 
parallel operation can occur over a significant period of time and 
needs to be carefully scheduled. 
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• The Systems Engineering Guidebook for Intelligent 
Transportation Systems (ITS) provides planning guidance for 
the retirement and replacement of large transportation systems. 
““Chapter 4.7 identifies several factors which can shorten the 
useful life of a transportation system and lead to early 
retirement, such as the lack of proper documentation, the lack of 
effective configuration management processes, and the lack of 
an adequate operations and maintenance budget”” (Pyster & 
Olwell, Product and Service Life Management, 2012). 

There is only one listed application for Enterprises and it is: 

• The disposal and retirement of large enterprise service systems 
requires a phased approach where capital planning is 
implemented in stages. As is the case of service systems, an 
enterprise system's disposal and retirement require parallel 
operation of the replacement system along with the existing 
(older) system to prevent loss of functionality for the users of the 
enterprise (Pyster & Olwell, Product and Service Life 
Management, 2012). 

The final section of this area discusses the need to recycle systems and 
the importance of that recycling not having an adverse effect on the environment. 
It also discusses the emerging area of “green engineering” and how important 
that concept and methodology is for the future of the environment and the planet 
(Pyster & Olwell, Product and Service Life Management, 2012). This concept is 
certainly becoming more important since the planet has limited resources and it 
is good practice to try and re-use materials whenever possible. This concludes 
the in-service portions of the SEBoK. 

5. Unique Features 

In addition to being the most comprehensive and lengthy of the industry 
standards that have been examined for this thesis the SEBoK also has a unique 
attribute and that is the revision plan. The editors of the SEBoK envision a 
constant revision an update process, with new versions coming out 
approximately every six months (Pyster, Olwell, & et-al, System Engineering 
Body of Knowledge, V 0.75, 2012). The method of revision involves accepting 
omissions, mistakes and suggested changes or updates from readers and users 

of the document (Pyster, Olwell, & et-al, System Engineering Body of 
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Knowledge, V 0.75, 2012). This constant revisions process should help to 
ensure the that the SEBoK captures to the most relevant information and 
techniques related to SE that are being used in academic and practical 
environments. The only major drawback that seems to be the potential for the 
editors to be overwhelmed by large numbers of submissions which can add to 
the length of time in-between updates. 

6. Summary 

The SEBoK has good coverage of Operations, Maintenance, Service Life 
Extension, Upgrades and Disposal processes. The only missing item from the 
SEBoK is an emphasis on the need for data gathering during the Maintenance 
period. The SEBoK is not the “best” at each section but the overall content is 
better written and taken as a whole presents a better product that the ISO 15288 
and INCOSE SE Handbook for in-service engineering items. The SEBoK still 
only provides a list of what needs to be done for in-service systems engineering, 
it does not say how to do something and that is what will be required by the 
NSEG. 
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IV. STANDARD COMPARISON 


A. NSEG VS. INDUSTRY STANDARDS 

1. Introduction 

As was stated in the previous chapter the NSEG does not have sections 
or information on in-service engineering. Because of this fact, all of the 
comparisons will be performed between the industry standards. From these 
comparisons, conclusions will be drawn about the importance of such sections 
and which standard is the best model for future revisions to the NSEG. 

Table 2 summarizes the findings from the four documents by listing the 
most important aspects of each of the in-service sections and noting how they 
address the material. If there is no discussion than “None” appears in the 
column, some coverage than “Partial Cover” appears in the column and finally if 
the subject is properly addressed “Cover” will appear in the column. Some of the 
factors leading into the coverage labels are word count, diagrams and specific 
points made. 



NSEG 

ISO 15288 

INCOSE 

SEBoK 

Operations 





Standards 

None 

None 

Cover 

Cover 

Data Gathering 

None 

None 

Partial Cover 

Cover 

Training 

None 

Cover 

Partial Cover 

Cover 

Customer Support 

None 

Partial Cover 

Partial Cover 

Cover 

Maintenance 





Standards 

None 

None 

Cover 

Cover 

Data Gathering 

None 

Cover 

Cover 

None 

Training 

None 

Partial Cover 

Partial Cover 

Cover 

Service Life Extension 





Cost 

None 

None 

None 

Cover 

Obsolecence 

None 

None 

None 

Cover 

Disposal 





Standards 

None 

None 

Partial Cover 

Cover 

Data Gathering 

None 

Cover 

Cover 

Cover 

Enviromental Concerns 

None 

Partial Cover 

Cover 

Cover 


Table 2. Comparison Standards Summary 
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Containing large amounts of content is not always an indicator of “good” 
coverage but it can give an indicator of importance placed on a subject. Table 3 
is a compilation of the number of words each standard has concerning a 
particular subject area. While length is not a substitute for content it is usually a 
good indicator something that the author/editor felt was important and thus added 
an appropriate level of content. 



NSEG 

ISO 15288 

INCOSE 

SEBoK 

Operations 

0 

669 

647 

1199 

Maintenance 

0 

590 

628 

746 

Service Life Extension 

0 

0 

0 

3538 

Disposal 

0 

652 

845 

1525 

Total In-Service SE 

0 

1911 

2120 

7008 


Table 3. Word Count for Standards 


2. Operations Summaries 

The ISO 15288 breaks the Operations down into three basic sections: 
Purpose, Outcomes and Activities and tasks. The Purpose is the definition of the 
“Operations Process,” the Outcomes are what you get as the result of a 
“successful implementation” of the process, and the Activities and tasks are what 
need to be done to achieve those outcomes. There are four desired outcomes 
and five activities and tasks that are listed and the activities and tasks each have 
one to three subtasks listed. This section contains some detail but is less than 
two pages for the entire amount of material for Operations (Roedler & Jones, 
2008). 


The INCOSE Handbook contains five sections: Purpose, Description, 

Inputs, Outputs and Process Activities. The Purpose is almost word for word the 

same as the Purpose in the ISO 15288. The Description explains exactly what 

the Operations process is and includes a useful diagram that summarizes all 

other sections of the Operations process (Figure 1 in Chapter III). The Inputs 

section describes all of the inputs, enablers and controls that feed into the 

Process Activities. The Outputs section describes all of the expected outcomes 

of the Operations Process and finally the Process Activities describe all of the 
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activities that get you to the Outputs with the aid of the Inputs, Enablers and 
Tasks. The final portion of the sections gives “common approaches and tips” 
and the entire section is three and a half pages long (Haskins, 2011). 

The SEBoK begins similarly to the INCOSE SE Handbook in that it begins 
with the definition of Operations from the ISO 15288 but it also utilizes the 
descriptions from the INCOSE SE Handbook in their “description” section. This 
first section also foreshadows the use of Service Life Extension (SLEP) and 
system Disposal as parts of the System Engineers job. This is one of three 
major sections, the other two being Process Approaches and Practical 
Considerations. The Process Approaches lists important data for Systems 
Engineers to gather and applicable tools and methods for the Operations 
Process and is intensive on data and explanation. The Practical Considerations 
section has a brief description and then lists the Outputs and Process Activities 
for the Operations Process. This section is a little over two and a half pages long 
(Pyster, Olwell, & et-al, System Engineering Body of Knowledge, V 0.75, 2012). 

3. Maintenance Summaries 

The ISO 15288 breaks the Maintenance Section into three sections: 
Purpose, Outcomes and Activities and Tasks. The Purpose is the definition of 
the “Maintenance Process,” the Outcomes are what you get as the result of a 
“successful implementation” of the process and the Activities and tasks are what 
needs to be done to achieve those outcomes. There are six desired outcomes 
and two sections of activities and tasks that are listed. The activities and tasks 
are broken down into Plan Maintenance and Perform Maintenance sections and 
each section has multiple subsections. This is sections contains some detail but 
is less than two pages for the entire amount of material for Maintenance (Roedler 
& Jones, 2008). 

The INCOSE Handbook contains five sections: Purpose, Description, 
Inputs, Outputs and Process Activities. The Purpose is almost word for word the 
same as the Purpose in the ISO 15288. The Description explains exactly what 
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the Maintenance process is and includes a useful diagram that summarizes all 
other sections of the Maintenance process (Figure 2 in Chapter III). The Inputs 
section describes all of the inputs, enablers and controls that feed into the 
Process Activities. The Outputs section describes all of the expected outcomes 
of the Operations Process and finally the Process Activities describe all of the 
activities that get you to the Outputs with the aid of the Inputs, Enablers and 
Tasks. The final portion of the sections gives “common approaches and tips” 
and the entire section is three and a half pages long (Haskins, 2011). 

The SEBoK System Maintenance section also begins with the definition 
from the ISO 15288 and then moves into the main considerations that need to be 
kept in mind during the process. The SEBoK then moves into its Process 
Approaches and Practical Considerations. The Process Approaches section 
covers the desired Outcomes, the Activities and Tasks and the Methods and 
Tools for the Maintenance Section. The Practical Considerations is rather limited 
and discusses changes in scope on Maintenance and how they should be 
handled. This entire section is less than two pages long (Pyster, Olwell, & et-al, 
System Engineering Body of Knowledge, V 0.75, 2012). 

4. Service Life Extension Summaries 

The ISO 15288 and INCOSE SE Handbook do not discuss system 
lifecycle extension and/or system upgrades. 

The SEBoK breaks extension and upgrades into two different sections. 
The SLE section begins with a description of key factors to consider and then 
defines those factors in the context of SLE. The next sections are how these 
particular key topics apply to Product Systems, Service Systems and 
Enterprises. Finally, the Practical Applications section ends with what is usually 
the limiting factor for SLE and that is the costs. This section is five pages long 
and contains extensive detail and reference material (Pyster, Olwell, & et-al, 
System Engineering Body of Knowledge, V 0.75, 2012). 
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The Capabilities Update, Upgrades, Modernization (CUUM) section 
begins with the definitions of what each of those items is, and what the possible 
reasons are for pursuing them. The next section discusses Key Factors and Key 
Processes and Procedures to consider. The next sections are how these 
particular key topics apply to Product Systems, Service Systems and 
Enterprises. The SEBoK then gives a “V” model example of how this can be 
represented and then discusses Practical Considerations. The ending Practical 
Consideration is a discussion on Commercial off the Shelf (COTS) items and the 
benefits and dangers associated with them. This section is five pages long and 
contains extensive detain and reference material (Pyster, Olwell, & et-al, System 
Engineering Body of Knowledge, V 0.75, 2012). 

5. Disposal Summaries 

The ISO 15288 breaks the Disposal Section into three sections: Purpose, 
Outcomes and Activities and Tasks. The Purpose is the definition of the 
“Disposal Process”, the Outcomes are what you get as the result of a “successful 
implementation” of the process and the Activities and tasks are what needs to be 
done to achieve those outcomes. There are five desired outcomes and three 
sections of activities and tasks that are listed. The activities and tasks are 
broken down into Plan Disposal, Perform Disposal and Finalize the Disposal 
sections and each section has multiple subsections. This is sections contains 
some detail but is less than two pages for the entire amount of material for 
Maintenance (Roedler & Jones, 2008). 

The INCOSE Handbook contains five sections: Purpose, Description, 
Inputs, Outputs and Process Activities. The Purpose is almost word for word the 
same as the Purpose in the ISO 15288. The Description explains exactly what 
the Disposal process is and includes a useful diagram that summarizes all other 
sections of the Disposal process (Figure 3 in Chapter III). The Inputs section 
describes all of the inputs, enablers and controls that feed into the Process 
Activities. The Outputs section describes all of the expected outcomes of the 
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Operations Process and finally the Process Activities describe all of the activities 
that get you to the Outputs with the aid of the Inputs, Enablers and Tasks. The 
final portion of the sections gives “common approaches and tips” and the entire 
section is three and a half pages long (Haskins, 2011). 

The SEBoK begins with the definition of System Disposal and with an 
emphasis on all of the environmental concerns that need to be addressed when 
the system is being designed. It utilizes the definition from the INCOSE SE 
Handbook and cites EPA and OSHA requirements for US products. The next 
three sections discuss applicability to Product Systems, Service Systems and 
Enterprises and further report various environmental and regulatory agencies for 
various countries and continents. The ending Practical Considerations section 
discusses the importance of “green engineering” and the long-term effects for the 
environment (Pyster, Olwell, & et-al, System Engineering Body of Knowledge, V 
0.75, 2012). 

B. INCOSE SE HANDBOOK VS ISO 15288 

1. Operations 

The INCOSE manual begins each of its sections with the definition of the 
particular area from the ISO 15288 so it would be safe to call it the parent 
manual. The reason given for revising some of the earlier versions to the 
INCOSE SE Handbook are to write an “Updated version based on ISO/IEC 
15288:2008; resubmitted to ISO/IEC for consideration as an ISO/IEC Technical 
Report” (Haskins, 2011). This commonality is maintained throughout the 
standards with the biggest difference being the level of detail placed into the 
INCOSE manual. 

The INCOSE manual has a similar number of Outcomes/Outputs to the 
ISO 15288 with the big difference being the amount of detail added to the 
INCOSE manual. The System Description only appears in the INCOSE manual 
and only the INCOSE manual has diagrams. The INCOSE manual places a high 
importance on standards, rules and regulations that need to be implemented. 
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Both manuals emphasize the importance of training and the creation of feedback 
loops for correcting procedures as more time is spent operating the system. 
Both manuals also point out the importance of supporting the customer for the 
creation of trust and because it aids in fixing issues that arise if all parties are 
working well together. Both manuals seem to do an adequate job for the 
Operations Processes. 

2. Maintenance 

The INCOSE SE Handbook has a similar number of Outcomes/Outputs to 
the ISO 15288, with the big difference being the amount of detail added to the 
INCOSE manual. The System Description only appears in the INCOSE manual 
and only the INCOSE manual has diagrams. The items that are readily 
noticeable as missing from the ISO 15288 are from the Controls section of the 
INCOSE manual and those include Applicable Laws and Regulations, Industry 
Standards and Project Directives and Standards. Both manuals emphasize the 
importance of maintaining records for historical and correction purposes, and 
both manuals emphasize the importance of proper tools, training and personnel 
to properly perform the Maintenance. Both manuals seem to do an adequate job 
for the Maintenance section. 

3. System Life Extension 

Neither the ISO 15288 nor the INCOSE SE Handbook discusses the 
areas of Service Life Extension or System Upgrades. 

4. Disposal 

The INCOSE SE Handbook and ISO 15288 have a similar focus for 
disposal in the emphasis on disposing to protect the environment. The INCOSE 
manual has a similar number of Outcomes/Outputs to the ISO 15288 with the 
major difference being the amount of detail added to the INCOSE manual. The 
System Description only appears in the INCOSE manual and only the INCOSE 
manual has diagrams. The items that are readily noticeable as missing from the 
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ISO 15288 are from the Controls section of the INCOSE manual and those 
include Applicable Laws and Regulations, Industry Standards and Project 
Directives and Standards. The INCOSE manual places a heavy emphasis on 
environmental and green disposal while the ISO 15288 just glances over that 
aspect. Both manuals emphasis the importance of having a defined disposal 
plan early on in the engineering process and both emphasis the importance of 
keeping good historical records on the process so that lessons can be learned 
and it can be proven that disposals were done properly. Both manuals 
successfully describe the Disposal process but the INCOSE manual would have 
to be defined as “better” because of the current importance for environmental 
and recycling concerns. 

C. ISO 15288 VS SEBOK 

1. Operations 

The SEBoK begins with a data analysis section that is not present in the 
ISO 15288. Both references emphasize the importance of training but the 
SEBoK goes into greater depth and detail on specifically what needs to be done 
at each step to make training effective and to ensure that proper tools and 
support are available for the operators. Both manuals discuss feedback loops 
and the importance of supporting the customer and both manuals have similar 
Outputs/Outcomes and activities which create them. Overall, it would seem that 
a combination of elements from both of these manuals would be needed to 
create a functional Operations section. 

2. Maintenance 

The first readily noticeable omission from the SEBoK that is written into 

the ISO 15288 is the importance of maintaining a history all maintenance issues, 

problems and corrective actions for revisions and similar projects in the future. 

The SEBoK contains a list of important considerations that are not listed in the 

ISO 15288 and these considerations place great importance on the maintaining 

of maintenance schedules in order to allow the product to complete its 

58 



Operations (which it cannot do if it is down for repairs, either scheduled or 
unscheduled). Both manuals emphasize the importance of proper training, tools 
and support for maintenance personnel to perform their jobs. Both manuals also 
emphasize the importance of having consistent maintenance procedures and 
timetables for repair so that products are fixed the same way each time. Both 
manuals perform an adequate job of pointing out the importance of Maintenance 
operations but a combination of both manuals would probably give the best 
solution because of the glaring omissions. 

3. System Life Extension 

There is no need for comparison since the ISO 15288 does not address 

SLE. 


4. Disposal 

The SEBoK has a similar focus to the ISO 15288 in the emphasis on the 
importance of properly disposing of the system to minimize the effects on the 
environment. The ISO 15288 uses general terms and ideas while the SEBoK is 
significantly more detailed especially in the case of different types of pollution 
and environmental concerns and agencies that affect disposal processes. The 
ISO 15288 also does not specifically address green engineering or break the 
processes down by category. The ISO 15288 does emphasize the importance of 
maintaining a concise set of records and history of what was done for both a 
learning point and to satisfy all legal considerations. The SEBoK emphasizes not 
only the environmentally safe disposal of products but also discusses the 
importance of recycling whatever can be used again. The SEBoK is once again 
much more detailed than the ISO 15288 on the topic of Disposal and would seem 
to offer the best solutions for inclusion in a manual. 
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D 


INCOSE SE HANDBOOK VS. SEBOK 


1. Operations 

The SEBoK and the INCOSE SE Handbook both recommend data 
gathering but the SEBoK has a more extensive and informative section on what 
to gather and why it is necessary to have. Both manuals also discuss training 
but the SEBoK has a more thorough section on this and greatly emphasizes the 
importance of proper training and the various areas that should be emphasized 
to ensure the appropriate training level is reached. The INCOSE manual does 
emphasize regulations, standards and laws more than the SEBoK does, and also 
places a greater emphasis on the customer/manufacturer feedback loop. 
Overall, it seems that combining these two manuals will get you a “best solution” 
for Operations. 

2. Maintenance 

The SEBoK begins with some important considerations that are not 
covered by the INCOSE SE Handbook but the INCOSE manual once again 
emphasizes documents, laws and regulations that are not found in the SEBoK. 
Both manuals emphasize the importance of having the proper tools and training 
for maintenance personnel, and the important of maintaining records to help 
facilitate improvement on those processes. Both manuals also emphasize the 
importance of scheduling, downtime and the importance of proper support. 
Aside from a few minor differences both of these manuals do a good job of 
covering the Maintenance processes. 

3. System Life Extension 

There is no need to discuss SLE in that the INCOSE SE Handbook does 
not have a section on it. 

4. Disposal 

Both the SEBoK and INCOSE SE Handbook place heavy emphasis on 

environmental protection and “green” disposal of items, and they both thoroughly 
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discuss the regulations, laws and applicable agencies that affect disposal. The 
SEBoK does a better job of actually pointing out which agencies are directly 
responsible but both lead you in the proper direction. Both instructions point out 
the importance of System Disposal being part of the initial design construction 
and not something that is an “afterthought” once the system is getting towards 
the end of its life and needs to be disposed of. The INCOSE manual even states 
that this process will need to be updated as changes are made to the system and 
to laws/regulations that might affect it. Overall, both of these manuals do an 
excellent job of explaining the Disposal process. 
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V. CONCLUSIONS & RECOMMENDATIONS 


A. INTRODUCTION 

1. Introduction 

Based on the inclusion of in-service engineering aspects in the ISO 
15288, INCOSE SE Handbook and SEBoK, it is safe to conclude that these 
aspects are important parts of Systems Engineering that should be included in 
future revisions of the NSEG. The in-service engineering areas that were 
discussed in this thesis cover a significant amount of time of a systems life and 
need to be covered by future revisions of the NSEG. It is important to note that 
all three of the SE references that were reviewed for this thesis only point out 
what needs to be done at certain points of the process. Details on how to 
conduct in-service systems engineering in a Navy context would still need to be 
developed in the NSEG. 

B. RECOMMENDATIONS FOR IMPROVEMENT TO NSEG 

1. In-Service Engineering 

In-service Engineering is an important aspect of SE and the four areas 
discussed here should be included in future revisions to the NSEG. An Aircraft 
Carrier might take 20 years to design and build, and then sail for 50+ years on 
the open seas (Federation of American Scientists, 2011). There will be a large 
number of changes over the course of the ship’s lifetime and using SE principles 
to govern those changes will make them more effective and efficient in the end. 
By not having a common Navy standard, projects will be at the mercy of Industry 
standards that while acceptable for the most part are also very different on some 
aspects. These differences can be very great for similar products made by 
different companies utilizing different standards. The following sections provide 
recommendations for each area of in-service engineering for future revisions to 
the NSEG. 
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2. Operations 

None of the three industry standards discussed in this paper completely 
satisfied the Operations Process. There were “good” and “bad” elements for 
each item, and taking that into account the recommended pieces for the NSEG 
would be: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• The five elements from this diagram and program would be 
populated from all three manuals. 

• Adopt the Data Gathering Suggestions from SEBoK, to provide the 
necessary data for studying various operations and functions so 
that improvements can be made for future iterations. 

• Adopt the emphasis on Customer Support and Feedback from ISO, 
as having good communication is vital to maintaining a healthy 
relationship between all involved parties. 

3. Maintenance 

There were many useful attributes described in each of the standards that 
would be useful to Naval Systems Engineers in a revision to the NSEG. The 
following suggestions are combinations of ideas taken from all three of the 
standards: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• The five elements from this diagram and program would be 
populated from all three manuals. 

• Adopt the emphasis on historical records from the ISO 15288 and 
INCOSE Handbook, as maintaining diligent records will aid in 
making changes and updates to maintenance procedures, tools 
and personnel qualifications. 

• Adopt the emphasis on training that is included in all three manuals. 

• Adopt the emphasis on consistency to establish a pattern for 
various Maintenance activities, this will allow new methods and 
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Operational changes to be made if necessary (i.e., more downtime 
than originally planned or retraining of Operational techniques 
because they are putting undue wear and tear on certain 
components). 

4. Service Life Extension 

The SEBoK is the only one of the three standards that addresses Service 
Life Extension and Capabilities Update, Upgrades and Modernizations so the 
recommendations will be drawn primarily from there with some formatting ideas 
taken from the INCOSE SE Handbook. This area should be broken down into 
two sections, first SLE: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• Adopt an emphasis on cost, as in order to extend a system it must 
be more cost effective than a replacement product in both the short 
and long term while performing maintenance. 

• Adopt an emphasis on obsolescence, to keep the spare parts, tools 
and qualified personnel on hand to keep the products running. 

Capabilities Update, Upgrades and Modernizations: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• Adopt an emphasis on cost, as in order to modernize or upgrade a 
system it must be more cost effective than a replacement product 
for the system would be, both short term and long term. 

• Adopt an emphasis on obsolescence. Are the spare parts, tools 
and qualified personnel on hand to keep the products running? If 
not then upgrades or modernizations may be required to keep the 
system running and/or in production. 

5. Disposal 

System Disposal seemed to gain the most interest from the standards 
because of the importance and ramifications that come from this portion of the 
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lifecycle. There were good elements from all three standards and the following 
sections of suggestions are a combination of those desirable elements: 

• Adopt an overall Model similar to the INCOSE one, with Inputs, 
Enablers and Controls feeding into Activities that give us Outputs. 
The diagram format that they use is informative from a summary 
purpose and should be included as well. 

• The five elements from this diagram and program would be 

populated from all three manuals. 

• Adopt the emphasis on environmental concerns for disposal and 
recycling that are present in all three manuals. 

• Adopt an emphasis on record keeping to preserve data for 

historical reference and as proof that products were properly 

disposed of IAW all applicable laws, standards and regulations. 

• Adopt an emphasis on making Disposal plans part of the process 

from the very beginning, and plan that they will be modified to 

address changes in the product itself and changing laws and 

regulations that govern it over the course of its lifetime. 

C. RECOMMENDATIONS FOR FURTHER STUDY 

1. Other Areas of the NSEG 

This thesis only focused on the in-service aspects of SE. There are many 
other attributes in the system lifecycle that need to be examined and studied to 
ensure that the NSEG is not missing any other relevant data, methods or tools in 
its processes. This would involve a similar methodology to what was employed 
in this thesis: first examine the NSEG to see what it has about a particular area of 
interest and then examine industry standards (either the ones used here or the 
ones discussed in the next section, ideally all) to see how they match up. This 
would be an involved and lengthy process due to the many areas that need to be 
covered and might in fact take several revisions to complete. 

2. Other Standards 

The ISO 15288, INCOSE SE Handbook and SEBoK were the three 
standards that this study was limited to, due to time and resource constraints. 
There are other standards for SE that might contain other data or ideas that 


66 



would be useful to Naval SE. Some of the other well recognized and utilized 
standards include the NASA/SP-2007-6105 (systems engineering handbook) 
(NASA Chief Engineer, 2007), the MITRE SE Handbook (MITRE Corporation, 
2012) and the ECSS E-10 (European Space Agency) (ECSS Secretariat, 1996). 
These standards all give similar information to the standards discussed in this 
thesis, but have different ideas approaches based on their contexts that could aid 
in development of a new version of the NSEG. 

In addition to reviewing standards, it would be good to have staffs working 
on this project that are well versed in several prominent systems engineering 
textbooks. These textbooks often give examples, models and techniques to 
follow in addition to presenting the general information that is found in standards. 
Some of the more widely used SE textbooks are “Systems Engineering and 
Analysis” (Blanchard & Fabrycky, 2010), “Control Systems Engineering” (Nise, 
2010), “Systems Engineering Principles and Practice” (Kossiakoff, Sweet, 
Seymour, & Biemer, 2011), “System Analysis, Design, and Development: 
Concepts, Principles, and Practices” (Wasson, 2005) and “Introduction to 
Systems Engineering” (Sage & Armstrong Jr., 2000)”. By combining these books 
and with the aforementioned standards there would be a more complete 
coverage of all areas of concern related to SE topics. 

3. Continual Revisions 

Another important point to make is that this NSEG update cannot be a 
onetime occurrence. There are new systems engineering tools and methods 
being developed all the time and only be performing periodic market surveys can 
these items be captured. The initial rework of the NSEG will be a time 
consuming and expensive process because of the long time since it was 
originally written, but subsequent revisions should be easier to accomplish as 
long as the periodicity is kept down. 

To promote this positive revision there should be a market survey 
performed every 3-4 years, that examines current revisions to industry SE 
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standards and determines if there are significant enough changes to warrant an 
update to the NSEG or certain sections of it. There needs to be an assigned 
Program Manger to this process to ensure that the surveys are done both timely 
and effectively. From these surveys the Program Office will need to determine 
whether or not a revision is warranted for the NSEG. 

D. CONCLUSIONS 

The NSEG is deficient in its discussion of the in-service areas of study 
including Operations, Maintenance, Service Life Extension, Capability Upgrades 
and Disposal. There is extensive information in the three industry standards that 
were examined for this thesis (ISO 15288, INCOSE SE Handbook and the 
SEBoK) and in several other standards that were mentioned in the 
recommended further research section of this chapter. Because these sections 
were not only included in these standards but also given significant coverage it is 
safe to assume that they are important to a successful system and should be 
included in future revisions to the NSEG. The industry standards for the most 
part contain the “what” of should be done so once the Navy decides which 
models it prefers they will have to fill in the “how” portions of the process. 

There will be a continually evolving threat to the United States and its 
allies and there will be both old weapons that are adapted to meet this threat and 
new ones that need to be created to respond to these threats (Greenert, 2012). 
By including in-service engineering in the NSEG and by continually updating its 
processes and procedures every few years the Navy can ensure that the NSEG 
is giving its work force the best tools from a systems engineering perspective to 
meet these new threats. This does not guarantee that the best services or 
systems will make it into the hands of war fighters but it will help to ensure that 
the best techniques are being used. These techniques will increase the 
likelihood that the Navy is getting the proper “bang for its buck” and that systems 
operate more smoothly and efficiently than those that do not utilize proper 
techniques. 


68 



APPENDIX 


SUGGESTED TABLE OF CONTENTS FOR NSEG REVISION: 

• System Operations 

• System Diagram with Enablers, Controls, Inputs, Activities 
and Outputs 

• System Operating requirements and parameters 

• Operator requirements and training 

• Operations data gathering requirements 

• Customer support/feedback requirements 

• Mission examination for equipment suitability, including 

submissions especially for larger platforms such as ships 
and submarines 

• Operations Execution 

• Differences between systems commands 

• System Maintenance 

• System Diagram with Enablers, Controls, Inputs, Activities 
and Outputs 

• System Maintenance requirements and parameters 

• Maintenance worker requirements and training 

• Maintenance data gathering requirements 

• Maintenance feedback requirements 

• Division of maintenance between ships force and shipyard 
level work (intermediate, long term, etc.) 

• Maintenance Execution 

• Mining maintenance data 

• System Service Life Extension 

• System Diagram with Enablers, Controls, Inputs, Activities 
and Outputs 

• Obsolescence issues including personnel, parts and tools 

• Cost analysis 

• Reliability analysis 
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• Mission analysis to include capabilities to address current 
and near future perceived threats 

• SLE Execution 

• System Capabilities Upgrade 

• System Diagram with Enablers, Controls, Inputs, Activities 
and Outputs 

• Obsolescence issues including personnel, parts and tools 

• Cost analysis 

• Resolving Integration Issues 

• Mission analysis to include “needs vs. wants” for current 
threats or perceived lifetime threats 

• Planned upgrades for “blocks” of ship or sub construction 

• Capability upgrade execution 

• System Disposal 

• System Diagram with Enablers, Controls, Inputs, Activities 
and Outputs 

• Environmental Concerns for System 

• Cost and Economic Analysis 

• Legal Concerns for System 

• Required Records 

• Recycling Requirements 

• Nuclear disposal requirements (if necessary) 

• Artificial reefing and or weapons target/demonstration (if 
appropriate), including proper environmental and cleaning 
requirements 

• Mothballing requirements (if necessary) 

• Formation of the Disposal plan 

• Modifications to the Disposal plan 

• Executing Disposal plan 


70 




LIST OF REFERENCES 


Bashaw, C. J., & Gardella, C. E. (1967). AFSCM 375-1. Bedford: AIR FORCE 
SYSTEMS COMMAND. 

Blanchard, B. S., & Fabrycky, W. J. (2010). Systems Engineering and Analysis 
(5th Edition) . Prentice Flail International Series in Industrial & Systems 
Engineering. 

Department of Navy. (2004). Naval Systems Engineering Guide. Washington 
D.C.: Department of Defense. 

ECSS Secretariat. (1996). Systems Engineering. Noordwijk: ESA Publications 
Division. 

Federation of American Scientists. (2011). CVN-68 Nimitz-class. Retrieved 
August 22, 2012, from Federation of American Scientists: 

http://www.fas.org/programs/ssp/man/uswpns/navy/aircraftcarriers/cvn68n 
imitz.html 

Federation of American Scientists. (2011). FFG-7 OLIVER HAZARD PERRY- 
class. Retrieved July 19, 2012, from Federation of American Scientists 
(FAS): 

http://www.fas.org/programs/ssp/man/uswpns/navy/surfacewarfare/FFG7_ 

oliverhazardperry.html 

Greenert, J. W. (2012, July). Payloads over Platforms: Charting a New Course. 
Retrieved August 22, 2012, from U.S. Naval Institue: 

http://www.usni.org/magazines/proceedings/2012-07/payloads-over- 
platforms-charting-new-course 

Haskins, C. (2011). INCOSE Systems Engineering Handbook: A GUIDE FOR 
SYSTEM LIFE CYCLE PROCESSES AND ACTIVITIES V 3.2.2. San 
Diego: INCOSE. 

INCOSE Administration. (2011). 2011 INCOSE Annual Report. San Diego: 
INCOSE. 

ISO. (2012, July). ISO Website. Retrieved July 11, 2012, from International 
Standards Organization: http://www.iso.org/iso/home.html 

JOINT OSD Working Group. (1993, September 7). MIL STD 499B (draft version). 


71 



Kockler, F. R., Withers, T. R., Poodiack, J. A., & Gierman, M. J. (1990). Systems 
Engineering Management Guide. Fort Belvoir: Defense Systems 
Management College. 

Kossiakoff, A., Sweet, W. N., Seymour, S. J., & Biemer, S. M. (2011). Systems 
Engineering Principles and Practice, Second Edition. Wiley. 

Massenburg, W. B., Langerich, A. W., Stone, D. T., Rodriguez, W. D., & Catto, 
W. (2004, October). MEMORANDUM OF UNDERSTANDING VS-MOU- 
20 . 

MITRE Corporation. (2012, June 26th). MITRE Systems Engineering Handbook. 
Retrieved August 22, 2012, from MITRE: 

http://www.mitre.org/work/systems_engineering/guide/ 

NASA Chief Engineer. (2007). NASA Systems Engineering Handbook. 
Washington D.C.: National Aeronautics and Space Administration. 

Nise, N. S. (2010). Control Systems Engineering, 6th Edition . John Wiley & 
Sons, Inc. 

Phillips, M. (2012, May 9). USS Nicholas Supports Drug Interdiction. Retrieved 
July 18, 2012, from US Navy: 

http://www.navy. mil/submit/display.asp?story_id=67063 

Pyster, A., & Olwell, D. (2012). Product and Service Life Management. In A. 
Pyster, D. Olwell, & et-al, Systems Engineering Body of Knowledge V 0.75 
(pp. 325-344). Hoboken, www.sebokwiki.org: Stevenson Institute of 
Technology. 

Pyster, A., Olwell, D., & et-al. (2012). System Deployment and Use. In A. Pyster, 
D. Olwell, & et-al, Systems Engineering Body of Knowledge V 0.75 (pp. 
267-279). Hoboken, www.sebokwiki.org: Stevenson Institute of 
Technology. 

Pyster, A., Olwell, D., & et-al. (2012). System Engineering Body of Knowledge, V 
0.75. Hoboken, www.sebokwiki.org: Stevenson Institute of Technology. 

Roedler, G., & Jones, C. (2008). Software and Systems Engineering-System 
Lifecycle Processes-ISO 15288. Geneva: International Organization for 
Standards. 

Sage, A. P., & Armstrong Jr., J. E. (2000). Introduction to Systems Engineering. 
Wiley. 


72 



Sheard, S. A., & Lake, J. G. (1998). Systems Engineering Standards and Models 
Compared. Herdon. 

Space & Missile Systems Center. (2005). SMC Systems Engineering Primer and 
Handbook, Concepts, Prinicples and Techniques, 3rd Edition. Space & 
Missile Systems Center. 

Task Force Devil. (2003). The Modern Warrior’s Combat Load: Dismounted 
Operations in Afghanistan. Coalition Task Force 82, Coalition Joint Task 
Force 180. 

Team Ships. (2012). SEA 21 Navy Inactive Ships Program-Ship Dismantling. 
Retrieved August 22, 2012, from Naval Sea Systems Command-Team 
Ships: 

http://www.navsea.navy.mil/teamships/lnactiveships/Ship_Disposal/default 

.aspx 

The Business Dictionary. (2012). Gap Analysis Definition. Retrieved July 18, 
2012, from The Business Dictionary: 

http://www.businessdictionary.com/definition/gap-analysis.html 

U.S. Naval Forces Southern Command and U.S. 4th Fleet Public Affairs . (2012, 
April 4). USS Elrod Seizes 1,000 Pounds of Drugs in Caribbean. Retrieved 
July 18, 2012, from US Navy: 

http://www.navy.mil/submit/display.asp?story_id=66281 

USAF Systems Command. (1974, May 1). Military Standard Engineering 
Management, MIL-STD 499A. 

USAF Systems Command. (1969, July 17). Military Standard Engineering 
System Management, MIL-STD 499. 

Wasson, C. S. (2005). System Analysis, Design, and Development: Concepts, 
Principles, and Practices. Wiley-lnterscience. 

Wynn, M. W. (2004, February 20). Policy for Systems Engineering in DoD. 


73 



THIS PAGE INTENTIONALLY LEFT BLANK 


74 



INITIAL DISTRIBUTION LIST 


1. Defense Technical Information Center 
Ft. Belvoir, Virginia 

2. Dudley Knox Library 
Naval Postgraduate School 
Monterey, California 

3. NAVSEA 5.0, Systems Engineering Directorate 
Naval Sea Systems Command 
Washington, D. C. 

4. NAVAIR 4.1, Systems Engineering Directorate 

Naval Air Systems Command 

Patuxent River, MD 

5. NAVSUP N6, Systems Engineering Directorate 

Naval Supply Systems Command 
Mechanicsburg, PA 

6. SPAWAR Science, Technology and Engineering Directorate 
Space and Naval Warfare Systems Command 

San Diego, CA 

7. MARCORSYSCOM SIAT 
Marine Corps Systems Command 
Quantico, VA 

6. Michael Frey 

Naval Sea Systems Command 
Washington, D. C. 


75 



